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ABSTRACT 
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This  report  presents  a  variety  of  fundanrental  invest! -  0 

gations,  with  common  analytical  foundation  in  the  work  © 
of  Jacques  Hatamard,  conducted  for  the  purpose  of  develop¬ 
ment  of  advanced  communications  systems.  Chapter  I 
presents  a  functional  analysis  approach  to^the  demodula¬ 
tion  problem^  as  formulated  in  a  system  of  integral  equa¬ 
tions  by  D,  C,  Youl'a.  Its  principal  content  is  the  dis9- 
cussion  of  the  engineering  significance  of  a  quadratic raj 
variable  principle  that  underlies  the  Youla  formulation. 
Chapter  II  concentrates  on  mathematical  considerations 
with  the  dual  purpose  of  completing  the  arguments  in¬ 
volved  in  Chapter  I  and  indicating  where  forward  work 
is  needed  to  enlarge  the  domain  of  the  validity  of  the 
theory  of  Chapter  I.  Chapter  III  presents  communica¬ 
tion  techniques  derivable  from  incidence  matrices  of 
balanced  incomplete  block  design  configurations© 
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INTRODUCTION  AND  SUMMARY 


The  research  on  communication  system  techniques  reported  in  this  document 
has  a  common  basis  in  analytical  concepts  introduced  by  the  great  French 
mathematician,  Jacques  Hadamard,  between  the  years  1893  and  1945. 

Chapter  I  presents  a  functional  analysis  approach  to  a  system  of  integral 
equations  introduced  in  1954  by  D.  C.  Youla.  Absolution  of  Youla's  equa¬ 
tions  yields  a  maximum  likelihood  estimate  of  continuous  modulated  intelli¬ 
gence  which  has  been  corrupted  by  noise.  The  basic  result  of  Chapter  I  is 
the  identity  of  Youla's  equations  with  the  first  variation  of  a  quadratic  func¬ 
tional.  In  order  to  devote  the  entire  first  chapter  to  this  basic  result  and 
its  consequences,  the  existence  of  certain  representations  needed  in  the 
formulation  of  the  quadratic  functional  is  postulated  gmd  their  actual  con¬ 
struction  is  deferred  to  Chapter  II. 

© 

Chapter  II  begins  with  the  construction,  under  suitable€bestrictions ,  of  the  Q 
representations  needed  for  Chapter  I.  An  attempt  to  lighten  the  restrictions 
so  as  to  include  all  cases  of  physical  interest  leads  us  to  the  conclusion, 
already  noted  by  Youla,  thl^  the  class  of  functions  to  be  considered  as  can¬ 
didates  for  solution ^iust  be  widened  to  include  distributional  solutions, 
i.  e.  ,  Dirac  delta  functions  and  their  derivatives.  The  question  of  preserv¬ 
ing  integral  representations  for  such  solutions  was  the  basic  consideration 
in  Hadamard's  invention  of  the  "finite  part"  of  a  divergent  integral.  This 
concept  toge^ier  with  Hadamard's  concept  of  a  "properly  posed"  problem  are 
introduced  and  shown  to  underlie  two  basic  approaches  of  importance,  namely, 
the  repre^lntation  bygiivergent  integrals  and  the  representation  by  Fourier 
integrals.  @ 

© 

Chapter  III  presents  communication  systems  techniques  derivable  from  in¬ 
cidence  matrices  of  balanced  incomplete  block  design  configurations.  For  a 
symmetric  design  the  above@;onfigurations  are  called  ^  %  configura¬ 

tions  and  these  constitute  a  generalization  of  Hadamard  Matrices,  i.e.  ,  for 
special  values  of  >.  there  exists  a  corresponding  Hadamard  Matrix 

and  conversely  every  Hadamard  Matrix  generateFa  configuration. 

A  Hadamard  Matrix  is  an  orthogonal  matrix,  all  of  whose  elements  are  it  1. 
For  several  years  engineers^  particularly  at  the  Jet  Propulsion  Laboratory, 
have  utilized  such  matrices  in  the  problem  of  the  optimum  codes  for  com¬ 
municating  through  space .  In  our  work  we  regard  the  incidence  matrix  of  a 


configuration  as  a  generalization  of  orthogonal  matrices;  i.e.  ,  we 
define  "almost  orthogonal"  matrices.  These  are  then  utilized  as  the  basis  of 
an  "almost  orthogonal  multiplexing  technique"  which  has  certain  attractive 
features  for  use  as  an  anti-jamming  system.  To  as  complete  an  extent  as 
time  permitted  we  investigated  questions  of  circuit  configurations,  timing, 
and  practical  implementation  for  rS'j  configurations  of  low  order.  From 
an  engineering  viewpoint  it  turns  out  that  the  increase  in  circuit  complexity 
in  going  from  an  orthogonal  system  based  on  a  Hadamard  Matrix  to  an  "almost 
orthogonal"  system  based  on  a  configuration  is  the  additional  inclu¬ 

sion  of  a  set  of  fixed  resistors  at  the  output  terminals. 

The  investigations  described  in  this  report  were  begun  in  April  1962  and 
concluded  in  March  1963.  The  work  was  performed  at  Vitro  Laboratories 
in  West  Orange,  New  Jersey.  The  principal  investigator  was  Dr.  R.  G. 
Segers, and  the  Rome  Air  Development  Center  Project  Monitor  was  Mr. 

Alfred  Kobos.  The  work  was  supported  by  Contract  AF  30  (602)-2649. 

©  © 

In  August,  1962,  the  project  monitor,  Mr.  Alfred  Kobos,  brought  to  our 
attention  the  work  of  Youla  which  resulted  in  the  treatments  in  Chapter  I 
and  Chapter  II.  In  October,  1962,  the  principal  Investigator  met  with  Mr. 

#  Kobos  and  Dr.  John  Lawton  of  the  Cornell  Aeronautical  Laboratory  in 
Buffalo,  New  York  tg>  discuss  Youla's  equations.  Dr.  Lawton's  approach 
to  Youla's  equations  is  based  on  an  extension  of  Youla's  series  technique  for 
the  FM'  case  and  does  not  overlap  our  approach  but  the  discussion  served 
to  focusfour  attdltion  on  a  functional  approach  and  the  consequent  use  of  the 
concepts  of  Hgdamard  that  were  available.  It  is  a  pleasure  to  expS’ells  at 
this  time  our  appreciation  of  the  technical  liaison  provided  by  the  project 
monitor,  Mr.  A.  Kobos,  and  of  the  stimulation  8i  the  resulting,  discussion 
with  Messrs.  Kobos  and  Lawton,  m  ® 


CHAPTER  I 

OPTIMUM  ANALOG  DEMODULATION  THEORY 


1.1  INTRODUCTION 


The  classical  procedure  for  minimizing  a  f miction  on  a  finite -dimensional 
space  is  to  set  its  gradient  equal  to  zero  and  to  examine  the  value  of  the 
function  at  these  points  and  the  signature  of  the  quadratic  form  whose  co¬ 
efficients  form  the  Hessian  matrix  of  the  function.  The  corresponding  pro¬ 
cedure  in  variational  problems  is  to  solve  the  Euler  equation  of  a  variational 
problem  and  to  test  the  solution  for  nature  of  the  extrema,  i.e.  ,  maximum, 
minimum  or  stationary  character,  by  the  study  of  the  second  variation.  On 
the  other  hand  when  an  equation,  irrespective  of  its  origin,  can  be  interpreted 
as  the  Euler  equation  of  a  variational  problem  one  can  often  obtain  algorithms 
for  the  solution  of  the  corresponding  functional.  Everything  we  do  in  this 
chapter,  and  indeed  in  most  of  the  next  chapter,  is  centered  on  examining  a 
set  of  integral  equations  whose  solution,  or  solutions,  would  determine  the 
optimum  receiver  (in  the  sense  of  maximum  likelihood)  in  the  presence  of 
additive  nSise  for  the  ca^se  in  ^hich  an  analog  signal  •x.i't)  is  transmitted  by 
means  of  a  carrier  which  is  modulated  by  «Uv>  in  some  prescribed  way.  The 
equations  were  introduced  by  Youla,  (sV'S  Youla,  page  95)  in  an  outstanding 
paper  in  1954.  Unfortunately,  in  the  cas^s  of  most  practical  interest,  namely 
phase  modulation,  and  frequency  modulation,  it  has  not  yet  been  possible  to 
solve  the  resulting  equations.  Consequently,  the  search  for  a  corresponding 
variational  problem,  which  has  been  successful,  is  detailed  below. 


The  case  of  optimum  demodulation  of  AM  signals  has  been  investigated  and 
an  approximate  solution  has  been  obtained  (40,  Thomas,  et  al. ).  A  maximum 
likelihood  estimation  process  for  FM  signals  has  been  investigated  by  line¬ 
arizing  the  integral  equations  of  Youla  (41,  Lawton,  page  11). 


We  introduce  notation  and  assumptions  compatible  with  Youla  (39;  Youla, 
page  90)  as  follows.  The  demodulator,  or  receiver,  is  assumed  to  operate 
on  the  last  T  seconds  of  data.  Symbolically  we  h^-e 


P 
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^Notation  relates  to  references  in  bibliography. 
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(1-1-1)  a.C'O  —  e-a_  <Uie>3  +•  avC^J)  ;  -t-~r  x  ±-b 

in  which  a. Lx)  and  /y^LX)  ,  both  statistical  in  nature,  represent  the  in¬ 
telligence  and  noise,  respectively,  and  Z.-JLX  i  «-  (.'ey  J  represents  a 

generalized  modulation  scheme.  For  example,  amplitude  modulation  (AM) 
may  be  designated  by  choosing 

(1-1-2)  AM:  ,c*Uv,  (  Ll0 X  +  (?) 

whereas  phase  modulation  (PM)  is  designated  by  choosing 


(1-1-3)  PM:  !>•'*+-  f  + 

and  frequency  modulation  (FM)  is  designated  by  choosing 


(1-1-4)  FM:  =  £0 


-t-r 


In  these  relations,  £c  is  the  amplitude  of  the  unmodulated  carrier  whose 
frequency  is  (aa  ,  cO  is  an  arbitrary  phase  angle,  and  X  is  a  modulation 
index.  Both  the  intelligence  a.iX)  and  the  noise  /*£&)  are  assumed  to  be 
Gaussian  processes  with  zero  mean  and  continuous  covariance  functions 
r*la_(s,T)  and 


© 

Having  received  the  waveform  ,  defined  in  (1-1-1),  the  ideal 

receiver  can  do  no  more  than  compute  the  a  posteriori  probability  density 
£  al/fc)  |  z,  (e>3  of  all  possible  intelligence  signals  <\.Lx)  (  3?  • 

Woodward;  p.  62.). 

© 

The  next  decision,  ana  this  is  a  critical  one,  is  to  decide  on  a  criterion  which 
will  produce  in  some  well  defined  sense  a  "best"  estimate  of  ctixy  , 
which  we  designate  by  a*(r}  •  The  criterion  adopted  by  Youla  (3^  ,‘  Youla; 
p.  93  )  is  the  maximum  likelihood"criterion  (  %  ;  Gramer;  p.  498  which 

chooses  such  that  ( 5Z-il~>3  is  maximized. 


We  now  have  enough  notation  to  write  down  the  celebrated  equation  of  Youla 
(3^  ;  Youla;  p.  95  )  gotten  by  a  derivation  that  we  will  discuss  at  a  later 
point  in  this  paper.  The  maximum  likelihood  estimate  (or  estimates) 

®  is  determined  by  the  following  two  integral  equations; 

(§>  ®  @ 

@ 


i 
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(1-1-5) 


rf 

<X*('C)  •=  \  ^  (Zjts  dtJ  -fc  -T  ^  t  ^  -fc 

J 


-t-T 


•b  -T 


-fc-'T^  'C  iL-b 


In  (1-1-5)  we  have  yet  to  identify  g  £5)  in  a  physical  context.  At  least 
to  this  author  has  no  natural  physical  interpretation  and  we  will 

treat  as  a  functi°n  whose  knowledge  will  aid  us  in  determining 

but  not  as  a  quantity  of  intrinsic  interest. 

Although  Youla  calls  this  a  maximum  likelihood  criterion,  it  may  be  termed 
a  maximum  a  posteriori  probability  criterion  since  the  a  priori  probability 
of  the  modulation  is  used. 

1.2  YOULA  EQUATIONS  AS  EULER  EQUATIONS 

Since  (1-1-5)  is  linear  in  ^  (X)  in  both  equations, we  first  solve  for 

and  manipulate  the  two  results.  We  postulate  the  existence  of  in¬ 
verse  kernels  (S.-tr)  and  (g  s\  such 

that  ^ 


(1-2-1) 


>  -t-Ti  £  f 


-t-T 


i  ^'v'-  ($js)ctS  —  -fe  -T  &  5  £  -t 

-t-T 

an  assumption  to  be  justified  later  in  the  discussion  in  Chapter  II.  Applying 
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(1-2-1)  to  (  1-1-5)  we  obtain 


(1-2-2) 


oJ*lT's><L'Z  ■=-  — 

£'~  "3  s. 


•i  -r 


■R-l  ^  e,(sj>  -  <t*Ls,«c*£5>J  J 


-i  -r 


Now  multiply  the  first  equation  in  (1-2-2)  by  S  ft.  (i)  >  an  arbitrary  func¬ 

tion,  and  the  second  equation  in  (1-2-2)  by 


(1-2-3) 


%  «.*( i) 

"S> 


then  integrate  both  equations  to  obtain 


b  -fc 


(1-2-4)  \  \  ^-L  (I,'*)  a*  (v)  %  O^Ct)  ftU  A.X.  = 


-t-r  -4-t 


-fc  -fc- 


(v)  *  ^  ^  ^ «-*^I  ~<zJ Sj^o] j 


4  -r-t-T 


t? 
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We  further  postulate  that  both  kernels  are  symmetric,  i.e.  , 


(1-2-5) 


A. 

(V)  -*  ^ j-  (syt) 


in  order  to  symmetrize  both  sides  of  equation  (1-2-4).  This  assumption  can 
be  by-passed  by  more  general  methods  ;  Tricomij  page  145),  which, 

at  this  early  stage  of  investigation  aren't  worth  the  resulting  complication  in 
manipulation  and  notation. 

We  now  define  the  quadratic  functional 

1 b  -fc 

(1-2-6)  ®  ^  (t^y  t+(sy  oPCt)4.l<LZ, 


a* 


and  the  limiting  process 


-t  -t  ~t~r 


(1-2-7)  -  JLL^  )  F  f-  £  S  a *cv}  -  Fjv*fe>l  ( 


£"»°  1  £ 


~t  -b 


o*(i)  4-  %  6?(sy*rct) | 


-i  -t  -t  -r 


-fc  -6 


-  ^  ^  *1  <fi.X 


-i-T  4-T 
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the  last  step  following  from  the  symmetry  of  >  i.e.,  from 

(1-2-5).  We  are  now  justified  in  using  the  notation  introduced  by  Volterra 
(_?>£.  ;  Volterra;  page  ^  as  follows, 

(1-2-8)  ~  \  3  F 

J  "a)  CC 

-fc-r 

which  is  a  generalization  to  a  continuum  of  variables  of  the  more  common 
formula 


vt 


(1-2-9)  <t  F 


-  2 


“3  F 


>c-< 


for  functions  of  /w-  variables  F  C'S  i  j _ ^  >  and  where  implicitly 

in  (1-2-8)  we  have  defined 


(i-2-io)  s  F  ~  ii'V 


*-T 


upon  comparison  with  (1-2-7).  In  this  notation  (1-2-4)  becomes 

(1-2-11)  -L  f  s  ds  ~ 


c> 


■fe-T 


-fcr  •*" 


'R-L  (tjS)  -  ga-HSA**Ul  ^  JZ.,  (5>  -  S  *(!:) 


-4-t  -tr-T 


Now  consider  the  functional  quadratic  in 


<2z.lv,>  » 


i.  e. ,  define 
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-v  -fc 


(1-2-12)  G  j_ & ■=  \  1  T£_l  (g.^) 


-fc-Tt-T 


and  the  limiting  process 
(1-2 


-is)  $g  =  ic-.v  ^  *•*<*.> i 

£-►0  /,  ?. 


■c  -t 


-  ^  ^  C$^)  g  hXst&bg 


-i-T  -fc-T 


+  3  ^ls,a*(s;3  s  a*(9  l  ,15^ 

3 


-t  -tr 


—  tL  ^  (^S)  ^  etls^o*^  %<&(&) JU&5 


-t-r  -t-r 


t  he  last  step  following  from  the  symmetry  of  (  ^_>  0  ,  i.e.  , 

from  (1-2-5).  We  are  now  justified  in  using  the  notation 


(1-2-14) 


^  A-/ 


S  «*&)  4.5 


-t-T 
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where  implicitly  in  (1-2-14)  we  have  defined 


(i-2-15)  s  c  is,  =  Tj_  c 


-i-T 


x  "3> 

In  this  notation  (1-2-11)  becomes 

-fc  . 

(1-2-16)  j_  r  3  p  tv  a-*tX>3  s  o?U)dA  — 

-t-T 

-fc  -t- 

-t-T  -t~T 

± 

'c)  £  L§  .  ll  %a*(0  A* 

^  <?c 

-t-T' 

To  further  simplify  (1-2-16)  we  define  a  linear  functional  in  £* X.^ 

by  t* 

(1-2-17)  ^  X  (^T>  fc*!?,  K*IV>1  cU<tr^ 

-t-T  -t-T 


3  Ct.)j  <L,(-C\  S  0.^0 


and  the  limiting  process 
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(1-2-18)  %  H  ■=  5  H  *  €-  s -  H  It 

£-^o  /  €-  J 


4:  -fc 

=  ^  j  R^Ll/t}.  ^  %a*(l)  t^dlcL'C 

-fc-T  -fc  -T 


We  are  now  justified  in  using  the  notation 

-o 


(1-2-19)  ^  H  7?x  (?,r)  *-2.1  § j g,(r\  J7_ 

J  ~  2>^ 

“t  -T' 

where  implicitly  in  (1-2-19)  we  have  defined 

(i'2_2o)  ?  hX.^  ^  r  g,  rv  9  c-.e^t/gx 

-?A,  J  ~ 

■i-T 


In  this  notation  (1-2-16)  becomes 

r.  + 

(1-2-21)  l  Lg  t  S  <x*(l)  Jji 

sj 

-i-T 


SlI  3  H 

J  ^ 

•i-T 


3  £  jj ,  e-xLT>  0-^)1  S  cifcyE 

"3  to 


-i-T 
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which  is  the  first  variation  ^  4  >  Courant  Vol.  I;  page  186  J  of  the  com¬ 

posite  functional 

(1-2-22)  X  -5  F  -  -a.  H 

at  the  function  fl.t'P)  ~  cv^C'fe)  ,  i.e.  ,  makes  (1-2-22) 

an  extremum.  To  see  this  analytically  we  note  that  the  first  variation  of 
(1-2-22)  is,  by  definition, 


(1-2-23)  VI 


O.’C  0*0*:} 


Ct  ■=«&*> 


-  o 

by  equation  (1-2-21).  Before  summarizing  our  results  as  a  theorem,  we 
combine  the  last  two  functionals  on  the  right  hand  side  of  (1-2-22).  This  is 
natural  since  G  is  quadratic  and  H  is  linear  in 

so  that  "completing  the  square"  produces  the  desired  result.  More  explicitly 
we  have  from  (1-2-22)  that 

•fr  - 1 

(1-2-24)  X  ^  K±.  (Sj^)  cUS)  a.Cb)  cLtd-Z 

■ fc  t 

•6-r-t-r 

-t?  t- 

-  ?L  Rx^t) 


-fc-r  -fc-T 


so  that 


■t  t 


(1-2-25)  X  I<U-*)3 


A.U)  CL<-t)  dlf^-C 


4-t  -fc-r 


Hr  -fr 


-t--T  -t—  I 


-fc-fc 

^-L  e,(  c)  6  ( (S)  c(f 

A*-  y 

-t-T-fc-T 


where  we  have  again  used  symmetry  of 
Since 


-U.  ^ 


(1-2-26)  *vO)  =  £,(r)  -  ^IS^^Lxy} I 


we  can  rewrite  (1-2-13)  as 
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(1-2-27)  ^  ^  Cl(l)^^)  U  cL'Z 

-fc-r-fc-r 


-b 


4  \  \  KJ-  U  "O  /W?)  ywC-r)  ^  l d'C 


-t- T  -fc-T 


-tr  -t 


I^L  (3.X)  £,^5;>  e,l^)  ctSJL-T, 

/VV  ./ 


■i'T  •fc-T 


In  summary,  the  above  manipulation  exhibits  a  quadratic  function, 

I  t;)JJ  in  equation  (1-2-27),  such  that  the  extrema  of 

X  HA-t'fc.hS  are  attained  by  the  same  a.*Cs)  that  solve  Youla's 

equations,  (1-1-5).  In  the  language  of  Variational  Calculus  the  Youla  equa¬ 
tions  are  the  Euler  equations  £  6,  ;  Courant,  Vol.  1;  page  184.).  We 

summarize  our  results  as  a  theorem,  the  discussion  of  whose  consequences 
constitutes  the  material  of  the  next  section. 


1 . 3  PRINCIPAL  RESULT  AND  CONSEQUENCES 


Consider  the  following  diagram. 
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vU«) 


ik(z) 


e-2.  Ll  ^ 

S. _ _ * 

Xfc  mod  , 

^  Mod. 

ft*  Ct> 


FIGURE  1  -  BASIC  BLOCK  DIAGRAM 

The  problem  is  to  design  the  demodulator;  i.e.  ,  knowing  £(  ('ey 
construct  ct*^) 

Theorem:  The  extremum,  i.e.,  the  maximum, 

minimum,  or  stationary  values  of  the  quadratic 
functional  ^  ^ 

(1-2-27)  X 

4-r+'T 
-tr  -k 

^  ^  dldJX. 

•b-T-h-T 


•b  Hr 


'fc-L  MO  <*,(**) 


■6-T-t-r 


Page  14 


are  provided  by  the  function  (or  functions)  Cx/^t' t)  for  which 


(1-2-23)  S  X 


Cl  ■=  0l¥('9 


~  S  X  1_cl  i~)  4-  £  S  R.tr>3  —  X 


«=.  O 


The  same  function  (or  functions)  a*(^  satisfies  the  equations 

f  Xx  (.^rj  a*(x)  dL-C  =•  £(S) 

^  Cl  _ 


(1-2-2) 


-t  -T 
tr 


••fc-T 

and, postulating  the  existence  of  inverse  kernels 
such  that 

^  'Rx.  (fj't)  d-'C  ■=■ 


(1-2-1) 


-t  — T 


ir 


^  Ri  (v)  R^tV)  =  S(«-*)  >  also  satisfies 

-fc-T 


the  equations  associated  with  the  maximum  likelihood  estimate  of  Youla 
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Y 


5 


(1-1-5) 


all  equations  being  valid  for  the  interval  and  symmetric 

kernels  •=.  (5 

Proof;  The  manipulation  involving  the  first  three  equations,  (1-2-27), 
(1-2-23),  (1-2-2),  of  the  theorem  are  straightforward,  though  lengthy,  and 
simply  amount  to  reversing  the  steps  taken  in  detail  in  Sections  1  and  2  of 
this  chapter.  The  manipulation  from  (1-2-2)  to  Youla's  equations  (1-1-5) 
involves  the  existence  of  inverse  kernels  so  as  to  give  validity  to  (1-2-1)  and 
is  essentially  an  application  of  the  Hilbert -Schmidt  Theorem  £  3^  ;  Tricomi; 

page  110).  In  essence  at  this  point  we  are  excluding  kernels  for  which  the 
Hilbert -Schmidt  Theorem  is  not  applicable  and  in  this  precise  sense  the  proof 
of  the  theorem  is  complete.  Since,  however,  there  are  kernels,  important  in 
applications,  for  which  the  Hilbert -Schmidt  Theorem  is  not  applicable  we  will 
discuss  the  necessary  additional  conditions  for  their  inclusion  at  a  later  point, 
namely,  in  Chapter  II.  Q.E.D. 

There  are  many  reasons  why  the  formulation  of  a  problem  on  a  quadratic 
variational  principle  is  important.  First  of  all  the  differential  equations  of 
mathematical  physics  have  been  successfully  analyzed  in  the  light  of  their 
relation  to  quadratic  varational  principles.  For  linear  functional  equations 
of  mathematical  physics  see  Courant£  £  5  Courant  Vol.  I,  page  252_)  and 

for  symmetrical  hyperbolic  systems  see  Courant  (_  7  >  Courant  Vol.  IIj 
page  592)  .  Secondly,  the  work  of  Synge£  3  I  ;  Synge  )  has  pointed  out 
the  advantage  of  a  geometrical  interpretation  of  a  quadratic  functional  in 
which  the  "distance"  of  the  approximations  to  the  exact  solution  can  be 
deduced.  Since  usually  the  engineering  problem  is  not  to  attain  the  exact 
solution  but  to  ascertain  that  a  structure  that  can  be  constructed  is  not  far 
from  the  best  possible  the  approach  of  Synge  could  turn  out  to  be  as  signifi¬ 
cant  for  communication  theory  as  it  presently  is  for  elasticity.  Thirdly, 
there  are  direct  numerical  techniques  for  establishing  upper  and  lower  bounds 


Youla;  page  95). 


-fc-T 

r * 

(t,(5>  -  l  K^tSjS) 

\-T 
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on  functionals.  A  definitive  modern  treatment  is  provided  by  a  translation  of 
the  Russian  "All-Union  Conference  on  Functional  Analysis  and  its  Applica¬ 
tions"  (  l  ;  American  Math.  Soc.  Trans.). 

Since  variational  principles  have  provided  some  of  the  greatest  generaliza¬ 
tions  in  all  of  physical  and  biological  science  it  is  appropriate  to  present  a 
short  history  of  the  attempts  of  early  natural  philosophers  to  discover  a 
minimizing  principle  in  nature.  The  first  such  discovery  was  the  "principle 
of  least  action",  where  action  is  to  be  understood  as  the  mean  value  of  the 
difference  between  the  kinetic  and  potential  energies  of  a  physical  system 
averaged  over  some  fixed  interval  of  time,  which  principle  originated  with 
P.  L.  M.  Maupertuis  (1698-1759).  Note  that  our  functional  in 

the  equation  (1-2-17)  contains  the  difference  of  two  integrals  with  the  same 
kernel  j'z-)  and  consequently  the  mathematical  form  for  an 

analogous  principle  is  available  for  speculation.  The  general  statement  of 
Maupertuis  was  made  in  an  attempt  to  extend  the  theorem  of  P.  Fermat 
(1601-65)  that  a  ray  of  light,  when  traveling  in  a  homogeneous  medium,  will 
pass  from  one  point  to  another  either  directly  or  by  reflection  by  the  shortest 
path  and  in  the  shortest  time.  Having  found  the  quantity  that  tends  to  a  mini¬ 
mum,  Maupertuis  regarded  the  principle  as  all-inclusive:  the  laws  of  move¬ 
ment  derived  from  it  would  apply  to  all  natural  phenomena.  The  astonishing 
degree  to  which  he  was  correct  in  his  prophecy  can  be  seen  not  only  in  modern 
physics  but  in  biology  as  well.  For  Maupertuis1  principle  is  none  other  than 
Claude  Bernard's  principle  of  the  maintenance  of  the  internal  environment, 
Walter  B.  Cannon's  principle  of  homeostatis, or  Le  Chatelier's  law  of  chemi¬ 
cal  equilibrium:  "In  a  system  in  equilibrium,  when  one  of  the  factors  which 
determine  the  equilibrium  is  made  to  vary,  the  system  reacts  in  such  a  way 
as  to  oppose  the  variation  of  the  factor,  and  partially  to  annul  it.  "  For  a 
fascinating  account  of  the  consequences  of  the  above  principles  see  Lotka's 
classic  work  on  mathematical  biology  (_  \ 7  j  Lotka;  page  152 )  . 

The  history  of  the  calculus  of  variations,  in  the  sense  of  obtaining  necessary 
and  sufficient  conditions  for  obtaining  extrema  of  integrals,  is  founded  mainly 
on  the  work  of  Euler  (1707-83)  and  Lagrange  (1736-1813)  with  Lagrange's 
work  extended  by  Hamilton  (1805-65)  in  his  classical  papers  on  dynamics. 
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1.4  THRESHOLD  PHENOMENA  IN  A  VARIATIONAL  SETTING 

One  of  the  dominant  ideas  in  modern  engineering  is  to  employ  "threshold" 
types  of  communication  systems.  In  essence^if  the  modulated  signal  has 
sufficient  power  relative  to  the  noise, a  virtually  perfect  transmission  is 
possible  C  V7  ;  Shannon;  page  9).  This  "all  or  nothing"  principle  is  natural 
to  quadratic  functionals,  indeed  it  appears  in  the  prototype  of  all  problems  in 
elastic  stability,  namely,  the  problem  of  the  buckling  of  a  long  slender  elas¬ 
tic  column  or  rod  under  compressive  forces  at  its  ends.  This  problem  was 
treated  some  two  hundred  years  ago  by  Euler.  The  physical  occurrence  is 
quite  easy  to  explain  and  understand.  Obviously,  if  the  column  is  long  and 
slender,  the  straight  unbent  state  of  it  will  not  remain  stable  if  the  end  com¬ 
pression  is  made  too  large:  that  is  clear  to  everyone's  physical  sense. 

What  is  interesting  to  us  about  the  problem  is  its  mathematical  description 
in  terms,  to  be  described  below,  of  the  difference  of  two  positive  quadratic 
forms  and  the  resulting  close  correspondence  of  the  functional  defined  by 
this  problem  with  X 

More  explicitly,  if  the  rod  or  column  is  of  length  L  and  if  its  lateral  dis¬ 
placement  is  denoted  by  tc(>9  .  0  fc  X  6.  U  j  then  the  potential  energy 

~0  is,  apart  from  material  constants,  given  by  (  L  ;  Courant  Vol.  I; 
page  273)  .  ^ 

(i-4-i)  -U'tjcCxxJ  ■=  ^  ^ 

The  first  integral  is  the  energy  of  bending,  the  second  integral  the  energy  of 
elongation,  so,  since  we  are  compressing,  the  negative  sign  appears. 


Lemma:  For  sufficiently  small  values  of  I  the  minimum  of  "V 

with  the  boundary  conditions  LJ)  *o  ,  has  the  value  zero. 

Moreover,  the  minimum  is  attained  only  by  IlO^)  50 

Proof:  For  example,  this  is  true  for  T.  <  U  •  Since 

(i  -4-2)  \  Tj  Cyc)  ds>C  —  7c  (o) 

o 

it  follows  that  there  exists  at  least  one  point  X-«  at  which 
(1-4-3)  ltCXo)^a  ^ 
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then  we  have  that 


(1-4-4)  ic  Og)  ^  C  tt ."Cxy 

%.o 

and  consequently 

(1-4-5)  ^  < 


<h<~  <.  \  W<*>] 


Since  (1-4-5)  is  true  for  O  &  X  —  ^ 


it  follows  that 

U 


Li  ^  Z.  0 

(1-4-6)  l  'L'U.'CxdZ  <X^c  <k  U  ^  W  V  I^0C3 

so  that  the  functional  “U*  t  icfxxl  defined  in  (1-4-1)  satisfies 


(1-4-7)  XT^U-ljoJ 


Tcu" 


+ 


f)L  7, 

\  E^'Cx^  ^  *>  0 


L~ 


Moreover,  equality  in  (1-4-7)  is  attained  only  for  =L-  O  .  This  is 

clear  from  the  last  term  in  (1-4-7).  Q.  E.  D. 


Taking  as  self  evident  the  principle  that  the  minimum  of  the  potential  energy 
n_T  yields  the  equilibrium  states  bcCk)  the  physical  meaning 

of  the  lemma  is  that  for  sufficiently  small  X  the  unbent  state  of  the  rod, 
itOc)  so  ,  is  the  only  equilibrium  state.  Since  it  is  clear  physically 
that  bent  states  tt 4  0  exist,  and  mathematically  for  any  such  admiss- 
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ible  states 


■uX>9  o 


we  can  choose 


(1-4-8)  X  > 


> 


we  see  the  possibility  that  °  .  From  the  lemma 

and  (1-4-8)  it  is  now  clear  that  there  exists  a  critical  value  of  H  ,.  call 
it  T0  ,  which  causes  buckling,  i.e.,  the  straight  state  ceases  to  be  the  only 
possible  equilibrium  state  and  a  bifurcation  of  the  solution  of  the  equilibrium 
problem  takes  place;  the  bent  state  is  then  the  stable  state  of  equilibrium. 
This  is  truly  a  bifurcation  phenomenon  in  the  sense  meant  by  Poincare  s 
Tricomi;  page  161^;  that  is,  let*)  «  /U.C>v  ^  »  a  function  of  both 
the  independent  variable  y.  and  of  the  load  parameter  IE  such  that  when  I 
attains  the  critical  value  R  new  types  of  solutions  of  the  equilibrium  pro¬ 
blem  appear  in  the  neighborhood  of  the  unbent  state  which  co-exist  for  the 
same  values  of  the  physical  parameters.  It  is  also  clear  that  is  the 

minimum  of  the  right  hand  side  of  (1-4-8)  for  U.ix)  .A  goodly 

number  of  the  readers  of  this  report  will  be  familiar  with  the  totality  of  re¬ 
sults  in  the  present  case  ;  Stoker;  page  251^)  ,  i.e.  ,  that  there  are  in¬ 
finitely  many  critical  values  and  correspondingly  many  different  buckled 
shapes  or  modes  of  the  column. 

What  can  the  above  discussion  add  to  our  -understanding  of  the  optimum  de¬ 
modulation  problem?  First  of  all  we  have  never  decided  what  type  of  ex¬ 
trema,  i.e.,  maximum,  minimum  or  stationary  point,  of  is 

appropriate.  The  above  discussion  yields  an  approach  that  indicates  a  mini¬ 
mum  is  appropriate.  To  see  this  consider  the  functional  > 

rewritten  here  for  convenience  as 
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(1-4-9)  TjL<a-C)J  ~  \  {  *-(t)  aL^  *r<*/c' 


■t  -r-fc-r 

■b  ~t 


4-  ^  ^  1,  il^>  -U^L-c. 


-t-r-t-r 
■fc  tr 


-u 


(,^U)  c.,(x.)  <£I<(X 


■4-T-t-T 


As  with  we  are  interested  in  the  reference  or  zero  state 

=  o  for  which  we  expect  the  estimate  o  .  Since 


(1-1-1)  <L\i'C}  ~  <Z~u  Lx,  ccCx^  -h  /v^trj 

we  get  from  (1-4-9)  that 
(1-4-10)  = 


b-b 

—  ^  'Rj.  it  pi)  t^6I  4iJL?- 

•h-T-b-b 

It  is  also  clear  that  for  any  reasonable  form  of  modulation,  say  the  AM, 

PM,  and  FM  cases,we  have  characterized  by  (1-1-  2),  (1-1-3),  and  (1-1-4), 
that  the  modulation  is  proportional  to  £e  ,  the  amplitude  of  the  unmodulated 
carrier.  Consequently  from  the  last  integral  in  (1-4-10)  we  can  factor  out 
bo  which  will  play  the  role  of  ~S.  >  o  in  our  previous  reasoning.  We 
need  one  more  general  fact  about  the  integrals  in  (1-4-9)  before  proceeding 
on  our  particular  tack.  We  want  to  establish  conditions  such  that  each 
integral  in  (1-4-9)  is  non-negative.  Fortunately  such  conditions  are  known, 
and  have  been  known  for  a  long  time. 


-t  b 

-t-T-t'T 
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Theorem  (Hilbert).  A  necessary  and  sufficient  condition 
for  the  quadratic  form  3"  ,  defined  by 

(1-4-11)  TC^f)  «  f1*)  , 

to  be  non-negative  in  —  space,  i.e.,  for  the  in¬ 

equality. 

(1-4-12)  TiQ*?)  'S  o  ,  dp  &  L.J. 

to  hold,  is  that  all  the  eigenvalues  of  be 

positive,  i.e., 

(1-4-13)  O  4  4  Xv  ^  ^3  4 - < 

Proof:  See  Tricomi(3>i/  ;  Tricomi;  page  124  ^  .  Q.  E.  D.  If  in  equation 
(1-4-12)  we  have  the  inequality  sign  whenever  is  non-zero  over  a  set  of 
finite  measure  the  kernel  KCx>^))  is  called  positive -definite. 

We  nowpostulate  that  the  kernels  K  L  C  and  P  i  (  S,  "t) 

are  positive  definite.  The  plausibility  of  the  assumption  is,  of  course, 
evident  from  (1-2-1)  which  defines  the  kernels  in  question  as  "inverses" 
of  the  covariance  functions  1?<x  (X'O  and  which  have  only 

positive  eigenvalues  and  moreover  are  positive  definite.  Precise  conditions 
for  the  validity  of  the  postulate  will,  as  in  our  basic  theorem,  be  discussed 
in  Chapter  II. 


We  now  return  to  our  reasoning  with  (1-4-10).  To  get  a  definite  point  of 
reference  for  X  La(/c)3  we  not  only  put  A.t'*)  =  o  but  turn  the 

carrier  power  down  to  zero,  i.e.  ,  from  (1-4-10)  we  have  that 


(1-4-14)  X 


—  O 


&0  =  O 

What  happens  as  we  increase  £o  from  zero  to  quite  a  large  value  in 
(1-4-10)?  The  first  integral  containing  /w (X)  depends  linearly  on 

£•2. H-'X'cl  and  hence  on  So  *  while  the  second  integral,  whose  con¬ 

tribution  we  know  to  be  positive  by  itself  and  negative  when  preceded  by  a 


negative  sign  as  in  (1-4-10),  depends  on  the  product  of  tt-jX'r*0- 


and 


u  LVl 

noise  A^(i> 
large 


and  hence  on  B?  .  For  a  fixed  but  arbitrary  sample  of 
it  is  clear  that  the  last  term  in  (1-4-10)  will  for  sufficiently 


make  I 


< 

o^C'c)  =  o 
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For  from  (1-4-10)  it  is  only  necessary  to  choose 


(1-4-15)  E0  •> 


-fc"  fc 


^  \  1  **-(%)  9-z.  <ij.  rL~C 


.fc  fc 


^  -iv  tVO  U>°3 


-t  -T-fc-r 


where  we  have  put 

(1-4-16)  -  E~' 

i.e.,  the  modulation  is  normalized  so  as  to  make  the 

role  of  E0  more  apparent.  There  is  a  close  correspondence  between 
(1-4-15)  and  (1-4-8)  since  the  fulfillment  of  each  implies  the  respective 
functionals  achieve  negative  values.  The  reason  for  assuming  the  kernel 
Rr  is  negative  definite  is  now  clear  from  (1-4-15)  since 

otherwise  the  possibility  of  the  denominator  vanishing  in  (1-4-15)  exists. 
Actually  this  assumption  is  not  that  critical  since  if  it  were  not  fulfilled  for 
some  'Rj.  (J.-fc)  of  interest  it  would  merely  be  necessary  to  choose 

the  modulation  Lt:  o.t'b) 3  so  as  to  avoid  the  null  function  of 

.  It  is  also  of  interest  to  comment  on  the  numerator  of 
(1-4-15).  Since  we  have  no  way  of  knowing  the  sign  of  avC?)  the  only 
reasonable  course  in  assigning  E.  ,  which  would  be  fixed  in  application, 
is  the  conservative  inclusion  of  absolute  value  signs  in  the  numerator  of 
(1-4-15). 


It  is  now  clear  how  to  handle  the  general  case  dlx)  .  Referring  now 

to  (1-4-  9)  instead  of  (1-4-10)  we  note  that  the  contribution  of  the  first  inte¬ 
gral  is  positive  and  hence  more  power,  i.e.,  a  larger  value  of  £«  ^  is  re¬ 
quired  to  make  ItAt'O]  <  0  .  Essentially  the  same  mechanism 

comes  into  play  as  with  (1-4-9).  The  last  integral  involving  <t,  =  t-i  ■+- 
is  capable  of  forcing  I  Lc'-t'5>3  <  <=  by  using  a  sufficiently  large  value  of 
So  .  We  summarize  our  results  as  follows. 


i 
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Theorems  For  fixed,  but  arbitrary,  samples  $.(•£)  and 

of  the  intelligence  and  noise  in  the  interval  -b -T  Si'S. 

-f  -4  "fe  and  for  positive  definite  kernels  "Rj. 
and  RLn  (.§>•&)  there  exits  a  sufficiently  large  value 
of  £<>  ,  the  amplitude  of  the  unmodulated  carrier,  to 
make  X  <  —  M  for  any  arbitrary  magni¬ 

tude  M  >o  . 

Proof:  The  term  in  X  which  is  quadratic  in  6«>  and  is  in¬ 

herently  negative  has  been  demonstrated  by  our  previous  remarks  to  dominate 
the  value  of  X][.a.C'5>3  •  Q.  E.  D. 


Equating  more  power,  i.e.,  larger  £*  with  better  transmission,  which 
by  the  above  theorem  is  equivalent  to  minimizing  ,  we  draw  the 

inference  that  for  fixed  power,  i.e.  ,  fixed  £0  .,  the. choice  a(r>  ■=.  , 

which  by  our  basic  theorem  gives  an  extremum  to  XE^J  ,  must 

correspond  to  a  minimum  of  X'C.tt.t'^3 


We  now  show  the  consistency  of  the  above  conclusion  with  the  basic  postulates 
on  which  Youla  has  based  his  theory.  Following  Youla  ('3')  ;  Youla;  page  93^) 
the  theorem  of  conditional  probabilities  is  that 

(1-4-17)  p(ccU,)  —  'j>lg»  I  °r.). - 

where  the  quantities  in  (1-4-17)  are  defined  by 


(1-4-18) 


•=  likelihood  of 

p(£<)  ^  likelihood  of  e.  (r) 


conditional  likelihood  of  «-p  L't.y  given 
Ci.('fc)  at  the  transmitter 


'pL'MC-i)  —  a  posteriori  likelihood  of  a.L'xy  given 
Z-t  at  the  receiver  . 


The  procedure,  called  the  maximum  likelihood  estimate,  adopted  by  Youla 
is  to  utilize  the  knowledge  of  to  maximize  ?<>  \Hi)  by  choice 

of  d-LXy  .  Youla  then  reasons  (  $9  ;  Youla;  page  94  J  that  since 

—  fcv  —  t*'  >  t  «->  ~  likelihood  that  e.,-  £ 
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noise^so  that  (1-4-17)  is  equivalent  to 
(1-4-19)  T>  e.)  -  1— ^ 

and  we  desire  to  maximize  o(cl|  • 


Lemma:  The  extrema  of  the  functional  are  attained  by  the  same 

function  <x*(fc*>  (or  functions)  that  yield  extrema  of  the  functionals 

H-Yff  1X10.1^3  ^  and  ^  —  T£&-(.t>3  3  since  both  mappings  are  monotone. 


Choosing  the  negative  exponential  in  the  above  lemma  we  obtain  from 
(1-2-27) that 


*tr  fc 


c: y  j>  \  “  \  Wu.  &«)  «-(. s)  42  4. z. 


-t-t-t-r 
■t:  fc 


(1-4-20)  «,*0 — 


x  e-Xff 

-i-T'-fc-T  J 


-fcfc 


CLyj  h  -  l  \  Rx  (t/e) 


-t-v-fc-V 


Compare  (1-4-20)  and  (1-4-19)  term  by  term  noting  the  correspondence  of 
our  double  integrals  with  the  quadratic  forms  in  equations  (8),  (9),  and  (12) 
of  Youla(ai  ;  Youla;  page  94^  •  Since  our  double  integrals  are  sometimes 
called  "quadratic  forms  with  an  infinite  number  of  variables  "(34 ;  Tricomi; 
p.  118^  the  correspondence  is  quite  close,  except  of  course  for  a  normal¬ 
izing  factor  to  make  probability  interpretations  possible.  Hence  the  right 
hand  sides  of  (1-4-20)  and  (1-4-19)  suggest  the  reasonableness  of  using  our 
functionals  in  place  of  the  series  technique  of  Youla.  The  correspondence 
of  the  left  hand  sides  requires  special  comment.  More  explicitly  we  have 
that 

(1-4-21)  (Lyty  |  _  r  ]  <> - *>  ^  O  U.) 

where  by  ^ - ?  we  mean  identical  except  for  a  normalizing  constant  factor. 

First  of  all  the  minimization  of  1  >  which  by  our  basic  theorem  and 
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previous  reasoning  in  this  section  is  accomplished  by  the  same  that 

satisfy  Youla's  equations  (1-1-5),  yields  a  maximization  of  (1-4-21)  which 
is  precisely  the  maximum  likelihood  criterion.  Secondly,  the  fact  that 

is  a  known  function,  as  indicated  by  the  notation  ,  played 

an  essential  role  in  the  manipulations  leading  us  to  .  More 

explicitly?even  though 

(l-l-l)  £,(-c)  -  <2-t.  LX)  +-  a-o  (Lt) 

and  consequently  depended  on  cU.'C)  ,  we  did  not  allow  for  any  variation  in 
when  we  formed  the  expression  ?.  %«.(.'&>  .  This  was 

correct,  of  course,  only  because  was  explicit  data,  known  at  the 

receiver,  and  our  search  for  a  critical  by  variation  over  all 

admissible  ojjt;')  had  to  be  compatible  with  the  fixed  nature  of  (."&> 

What  we  are  saying,  of  course,  is  that  the  correspondence  (1-4-21)  is  very 
satisfying  in  the  number  of  checks  it  yields  on  the  correctness  of  the  theory 
of  this  chapter. 


1.5  SOME  SPECULATIVE  CONSIDERATIONS 

We  now  proceed  with  one  of  the  author's  favorite  ways  of  turning  an  intract¬ 
able  problem  into  an  at  least  approachable  problem  which,  hopefully,  bears 
resemblance  to  the  original.  First  of  all  we  introduce  a  geometry,  which 
will  again  require  the  assumption  that  any  relevant  kernels  like  R  U  ^  “51 
and  be  positive  definite  so  that  a  metric  can  be  defined.  As  in 

the  previous  section  the  discussion  of  the  conditions  for  which  the  assumption 
is  valid  will  be  deferred  until  Chapter  II.  Having  a  geometry  we  then  employ 
the  trick  of  assuming  that  the  solutions  of  interest  have  certain  simple  fea¬ 
tures.  For  example,  in  what  follows  we  construct  a  triangle  and  there  are 
certain  heuristic  reasons  for  asserting  the  triangle  is  a  right  triangle.  The 
resulting  simplification  in  analysis  is  considerable  so  we  proceed  with  opti¬ 
mism.  Finally  a  simple  structure  emerges,  but  we  must  change  the  setting 
of  the  problem  somewhat  in  order  to  obtain  a  consistent  structure.  The 
reader  who  is  scornful  of  such  tinkering  or  who  feels  no  need  of  geometrical 
motivation  may  proceed  at  once  to  Chapter  II  without  any  loss  in  logical 
continuity. 

In  order  to  associate  a  geometrical  picture  with  we  introduce 

a  vector  notation. 

(1-5-1)  It  x.  A Ci)  ^  j  C  •  e,(5>^  1 
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and  the  inner  products  appropriate  to  the  kernels  1?  J,  (? >x )  and 

7Ll  ,  i.e. ,  put 


-fc  "b 


^  IZa-Cz,*)  Xts)  ^  (t)  oil  JL'C 


(1-5-2) 


•b-T’-fc-T 


■tr  Hs- 


'Rj- 


-t-r-t-T 


For1  either  inner  product  in  (1-5-2)  we  introduce  the  notation 

(■-5-3>  tmi  =  fUTzJ1 


which  is  the  distance  from  X-  to  the  null  vector  &■  at  the  origin  of 
the  coordinate  system.  Then  from  (1-2-27)  we  obtain  in  the  above  notation, 

(i-5-4)  iUm  ~  |\^I|L  •+-  -  lltTiu 

Since  the  data  received,  C-il'b)  ,  is  an  additive  combination  of  the  modulated 
signal,  ,  and  the  noise,  a-vCx)  ,  we  can  write 


(1-5-5) 


e-i  - 


4-  Av 


Inserting  (1-5-5)  in  (1-5-4)  we  obtain  another  expression  for  T  JpLfx'Tj 
namely, 

(1-5-6)  1  LccCxyJ  ^  tt  ^  Hi  +I!^!|_l  — 
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Note  that  (1-5-6)  is  identical  to  (1-2-22)  which  is  the  form  of  our  functional 
before  "completing  the  square",  accomplished  by  equations  (1-2-22)  to 
(1-2-27). 


Having  before  us  the  simplest  possible  representations  for  , 

namely,  (1-5-4)  and  (1-5-6)  we  ask  what  choice  of  cxtx>  •=, 

can  lead  to  an  extremum  of  X  £0X193  ?  The  last  term  in  (1-5-6) 

yields  a  clue  to  a  contribution  to  that  we  cannot  control  by  choice 

of  oXt:)  ■=.  .  Recalling  that  our  knowledge  of  ^  »  ewU), 

the  noise  being  added  to  our  modulated  signal,  is  embodied  in  the  kernel 
<£/w(s,'c)  ,  which  is  the  covariance  function  of  the  noise,  assumed  to 

be  a  continuous  second-order  process  with  mean  zero  (  ^  ;  Youla;  page 
92),  it  follows  that  the  numerical  sign  of  in  the  term  j 

cannot  be  known  to  us.  This  is  a  basic  point.  Having  assumed  that  the 
processes  of  interest  have  mean  zero  and  having  characterized  our  problem 
by  covariance  functions,  which  are  expected  values  of  the  product  of  two 
processes,  we  have  no  possibility  of. knowing  the  numerical  sign  of  the 
noise  contribution  in  the  term  (  eT  -J-  .  Note  that  all  other  terms 

in  (1-5-4)  and  (1-5-6)  are  quadratic  and  hence  insensitive  to  the  numerical 
sign  of  <x.  }  y  o/l  T,  .  .  Since  the  term  (e.*..,*-)  cannot  be 

controlled  relative  to  yielding  an  extremum  of  xto.t/e>3  ,  it  is  of 
interest  to  investigate  the  consequences  of  assuming  that  ^  -  0 

Even  more  to  the  point  is  the  question  of  the  consistency  of  the  condition 
(  -  0  with  the  solution  (or  solutions)  of  Youla1  s  equations. 

Before  plunging  into  the  particular  question  posed  by  the  last  paragraph  some 
general  remarks  are  in  order  to  achieve  the  proper  perspective.  In  the 
theory  of  ordinary  maxima  and  minima  the  existence  of  a  solution  is  ensured 
by  the  fundamental  theorem  of  Weierstrass(C»  ;  Courant;  page  57).  In  con¬ 
trast,  the  characteristic  difficulty  of  the  calculus  of  variations  (and  of  formu¬ 
lations  like  Youla's  which  our  Theorem  shows  is  equivalent  to  a  question  in 
the  calculus  of  variations)  is  that  problems  which  can  be  meaningfully  formu¬ 
lated  may  not  have  solutions.  In  essence  it  is  not  in  general  possible  to 
choose  the  domain  of  admissible  functions  as  a  "compact  set"  in  which  a 
principle  of  points  of  accumulation  is  valid.  This  is  best  illustrated  by  the 
following  geometric  example(6  ;  Courant,  Vol.  I;  page  173)  .  Two  points 
on  the  x-  axis  are  to  be  connected  by  the  shortest  possible  line  of  continuous 
curvature  which  is  perpendicular  to  the  x-  axis  at  the  end  points.  This  pro¬ 
blem  has  no  solution.  For,  the  length  of  such  a  line  is  always  greater  than 
that  of  the  straight  line  connecting  the  two  points,  but  it  may  approximate 
this  length  as  closely  as  desired.  Thus  there  exists  a  greatest  lower  bound 
but  no  minimum  for  admissible  curves. 
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Consequently,  in  the  calculus  of  variations  the  existence  of  an  extremum  in 
a  particular  problem  cannot  be  taken  for  granted.  A  special  existence  proof 
is  needed  for  the  solution  of  each  problem  or  class  of  problems.  Such  exis¬ 
tence  proof  s  have  not  been  published  relative  to  the  equations  of  Youla  (1-1-5). 
However,  an  indication  of  how  such  proofs  can  be  accomplished  is  contained 
in  the  work  of  Krasnosel'skii  and  Rutickii^/6  s  Krasnosel'skii;  Chapter  IV) 
on  convex  functions  and  Orlicz  spaces  (i.e.  normal  spaces  of  which  the  U  — 
spaces  are  a  special  case).  Since  the  Youla  equations  are  expressible  as  the 
gradient  of  the  functional  X  3  and  a  number  of  theorems  (  /  i  ; 

Krasnosel'skii;  page  214  )  on  the  existence  of  characteristic  functions  for  such 
a  gradient,  called  a  "potential  operator",  are  known,  it  may  be  possible  to  es¬ 
tablish  the  existence  of  solutions  of  the  Youla  equations  by  means  of  the  avail¬ 
able  theorems.  Such  an  effort  in  the  future  is  contemplated  by  the  author  as 
part  of  a  continuing  interest  in  this  area  of  research. 

The  point  of  the  above  discussion  is  that  there  may  or  may  not  exist  solutions 
of  Youla 'a  equations  or  equivalently  extrema  of  the  functional  X  L 
since  the  existence  of  a  bound  does  not  imply  the  existence  of  functions  attain¬ 
ing  the  bound.  Moreover  the  additional  condition,  =  o  ,  that 

we  have  reasoned  from  (1-3-6)  as  desirable, may  not  be  attainable.  Regard¬ 
less  of  these  possible  negative  results  we  now  investigate  the  consequences 
of  assuming  that 

(1-5-7)  -O  ~  ° 

Combining  (1-5-5)  and  (1-5-7)  in  a  geometric  construction,  using  the  metric 
n  iu  as  the  basis  of  length,  we  obtain  the  following  figure. 

'VC 


FIGURE  2  -  GEOMETRY  OF  RECEPTION  PROCESS 
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To  construct  the  above  figure,  called  a  hypercircle  by  Synge  (34  ;  Synge; 
page  87  )  ,  the  received  signal  CZ.,  is  surrounded  by  a  circle  of  radius 

U  U  x  ,  the  magnitude  of  ||  Tt",  II  x  being  assumed  larger  than 

the  assigned  radius  so  that  the  origin  of  coordinates  ©•  lies  outside  the 
circle.  From  the  origin  &  we  draw  the  tangents  to  the  circle.  Of  the 
two  possible  constructions  we  arbitrarily  choose  the  one  in  the  above  figure. 
The  essence  of  the  construction  is  embodied  in  the  single  condition  (1-5-7). 
From  the  figure  we  see  two  interpretations  of  this  condition.  First  of  all, 
the  modulation  has  been  positioned  orthogonal  to  the  noise  ^  which 

is  what  we  have  been  postulating  all  along.  A  second,  and  fresh  point  of 
view,  is  that  for  the  assigned  value  of  ||  l(  jl_  the  angle  &  which  sep¬ 
arates  (t,  and  has  been  maximized.  We  are  now  in  a  position  to 

draw  a  surprising  conclusion  concerning  © 


Theorem:  The  condition  of  orthogonality^ 
(1-5-7)  (  *-z.  ?  )  jL^  -  o  i 

suffices  to  determine  the  angle  •-  which 
separates  and  e_a,  .  More  ex- 

plicity  &  ■=  loo  . 


Proof:  Put  (1-5-7)  into  (1-5-6)  to  obtain  (1-5-8) 

(1-5-8)  It^)]  ~  ll  II  J_  — 

We  now  retrace  our  steps  to  obtain  the  equation  satisfied  by 

(1-5-9  £  T  [  *  (ty]|  =  0 

'  o.('t)  ^  {y) 


From  (1-5-9)  we  obtain 
(1-5-10)  SX  ^  CL 


Kl  %&*(*)  JLZ<Lt 


-fc-r  +--r 


-  X 


-fcf  -fc 

tj_  (?,S)  -O  gy  U ,  %A)  <LI <V> 

-i-T-fc-'T 


o 
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Since  %  is  arbitrary  we  obtain  from  (1-5-  10)  that 

(1-5-11)  \  Kc  C e*Ls,**(«Us 

1°-  \  "*• 


-t-r 

Comparing  (1-5-11)  with 


4  — T 


(1-2-2) 


•b 

)  ?t  (X^  <L*(^)  A-c  -  ^ S1 

-fc-r 

J  Xx  (%s>  |  £,(*>-  As  - 

-fc-T 


we  obtain  two  expressions  for  9-(s)  >  namely 


(1-5-12) 


$U>  —  ^  o^ls^s 

-t-T 

r* 

<^(%y  -=.  j  ^  -l  (?, *t)  f,  0  i  ( s)  —  £z  As 

-t-T 


From  (1-5-12)  we  have 

(1'5'13)  ^S,s)  e*, Is^/s)]  As 

-t-r 


^  a,  is)  As 

-t-r 
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Multiply  both  sides  of  (1-5-13)  by  2* -lJ>)  and  integrate  to 

obtain 

(1-5-14)  lltlli"  =  (tl^ju  =  r  JL 

Multiply  both  sides  of  (1-5-13)  by  Q~t  and  integrate  to  obtain 

(1-5-15)  -  (eT;e“,)-L  -  IU,  Hx. 

^  A*-  /U-  -K. 

F rom  (1-5-14)  and  (1-5-15)  we  conclude  that 

U-5-16)  ||  tint  =  (r  lit,  III 

A*.  1  -W. 

and  since  II  u  is  inherently  positive  that 

a-5-17)  ii  tin  j.  =  j-  h  t;  iu 

/V\_  /Vv 


There  is  an  elementary  theorem  in  Euclidean  geometry,  well  known  to  every 
school  boy,  that  states  "In  a  right  triangle  where  the  short  side  is  one  half 
the  hypotenuse  the  angles  are  30°,  60°,  90°.  "  Hence,  in  Figure  2,  ©=.60° 
and  the  sides  of  the  triangle  are  in  the  proportions. 

(1-5-18)  II  T)  IU  :  II  «\-ll  _|_  :  fl^llk  ■=  I  '•  i  •'  X 

AA 

Q.  E.  D. 


Now  that  we  have  arrived  at  a  definitive  ratio  of  metrics  in  (1-5-18)  it  is 
natural  to  attempt  to  "pack"  the  space,  whose  metric  is  determined  by 

II  ||  r  ,  with  triangles  of  the  form  of  Figure  2,  whose  sides  are  compatible 
with  (1-5-18).  Except  for  the  special  case  given  below  the  following  fundamental 
difficulty  is  encountered.  The  geometry  of  the  space  whose  metric  is  determined 
by  II  H  is  the  geometry  of  Hilbert  Space.  This  is  easily  seen  by 

comparing  our  construction  of  quadratic  functionals  with  the  treatment  of  recip¬ 
rocal  quadratic  variational  problems  as  given  by  Courant(^£s  ;  Courant,  Vol.  I; 
page  252)  .  Since  Hilbert  Space  is  infinite -dimensional,  i.  e.  ,  for  any  integer 
/w  one  can  find  in  Hilbert  Space  /vx  linearly  independent  vectors, the  constraint 
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in  (1-5-18)  and  consequently  the  _30 ° ,  6_0°,  90°  angle  relation  is  not  sufficient 
to  pin  down  the  construction  of  and  /v~  unless  we  postulate  some 

further  condition.  Again  following  our  philosophy  of  seeking  the  simplest  solu¬ 
tion  compatible  with  the  conditions  of  our  problem  we  now  restrict  our  atten¬ 
tion  to  a  structure  that  will  produce  a  plane  in  the  Hilbert  Space  of  interest. 

See  Figure  3. 


What  we  have  done  is  restrict  our  possible  inputs  to  three  distinct 

functions  ,  Cill>  L't-}  ,  and  and  postulated  that  the  resulting 

modulated  signals  Z-2  E/tj  and 

(2^  Xst,  make  angles  of  120°  with  each  other.  Each  modulated  signal  is 
of  the  same  magnitude  and,  subject  to  the  planar  restriction,  is  separated 
as  much  as  possible  from  the  other  modulated  signals.  From  the  point  of 
view  of  coding  we  would  designate  the  modulated  signals  as  a  maximal  code 
(  3.7  ;  Wolfowitz;  page  78  ^  with  the  decoding  system  consisting  of  the  par¬ 
titioning  of  the  plane  by  the  dotted  lines,  each  of  which  is  the  negative  exten¬ 
sion  of  one  of  the  modulated  signals  making- an  angle  of  60°  with  the  other  two 
modulated  signals.  To  decode,  or  demodulate,  a  received  vector  <2.,  the 
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partitioning  of  the  plane  assigns  the  nearest  modulated  noiseless  signal,  i,e.  , 
either  ^  ,  or  a.,,  ^  from  which  the  original  is  obtained 

as  one  of  the  functions  cUb?  ,  a  (.">->  ,  or  «-('=>  .If  fc,  happens  to 

coincide  with  one  of  the  dotted  lines  then  there  are  two  modulated  signals  at 
the  same  distance,  either  of  which  may  be  used  without  increasing  the  proba¬ 
bility  of  error. 

In  summary  of  the  construction  we  note  that  our  search  for  a  simple  geometry 
has  required  consideration  of  a  discrete  set  of  signals  a.t'&S  so  that  the  modu¬ 
lations  might  all  lie  in  an  easily  visualized  plane.  Moreover,  the  desired  con¬ 
dition  of  orthogonality  (1-5-7)  and  the  consequent  angle  &  —  60°  are  only  sat¬ 
isfied  by  the  e.,  which  fall  on  the  lines  which  partition  the  plane  into  pie 
shaped  wedges.  From  a  heuristic  point  of  view  one  could  reason  that  only 
the  vectors  which  partition  the  plane  need  be  identified  in  an  optimal  fashion 
and  consequently  in  this  sense  the  problem  has  been  satisfactorily  handled. 
From  a  logical  point  of  view  much  remains  to  be  done  to  give  substance  to 
the  above  approach. 
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CHAPTER  II 

BASIC  CONCEPTS  OF  HADAMARD  AND  THEIR  APPLICATION 

2.  1  INTRODUCTION 

The  subject  matter  of  this  chapter  has  application  to  specific  questions  raised 
in  treating  Youta's  integral  equations  (1-2-27)  and  to  the  general  question  of 
integral  equations  of  the  first  kind  Ch  ;  Tricomi;  page  143^  with  symmetric 
kernels.  We  proceed  from  the  specific  to  the  general.  First  of  all, in  proving 
our  principal  result  concerning  Youla's  equations  (1-2-27),  i.e.,  the  identity 
of  (1-2-27)  with  the  first  variation  (1-2-23)  of  a  quadratic  functional  (1-2-27), 
we  assumed  the  existence  of  kernels  1 ^£_(t,~)  and  lExfj^x} 

defined  by  (1-2-1).  The  validation  of  this  assumption  is  absolutely  essential 
to  the  basic  ideas  in  Section  I  and  we  shall  correspondingly  spend  much  effort 
on  this  topic  in  what  follows.  A  second  but  less  critical  assumption,  since  it 
can  be  by-passed  to  some  extent  (see  the  discussion  following  (1-4-16),  is  the 
postulate  that  %ju  CZ,-r)  and  'Ex.  (J/e)  are  positive  definite. 

Cv  /V'~ 

(For  real  noise  n(t)  and  information  a(t),  the  kernels  are  symmetric  and  thus 
so  would  be  the  inverse  kernels. ) 

Our  organization  is  as  follows.  Since  the  construction  of  the  kernels 

and  follow  the  same  pattern,  we  choose 

one  of  them,  in  particular  Rj ,  and  discuss  it  in  complete 

detail.  This  will  validate  the  assumption  of  Section  I  and  show  us  that  our 
theory  is  not  vacuous.  We  then  observe  the  desirability  of  alternate  conditions 
for  validation.  The  point  is  that  our  conditions  for  validation,  though  suffi¬ 
cient,  may  be  awkward  when  applied  to  a  particular  problem.  -This  point  of 
view  for  integral  equations  of  the  first  kind  has  been  put  forward  by  Shinbrot 
(iff  ;  Shinbrot)  page  3 )  and  when  our  kernels  are  difference  kernels,  corres¬ 
ponding  to  stationary  statistics,  Shinbrot(zy  ;  Shinbrot;  page  4jj  has  an  al¬ 
ternate  approach  based  on  the  use  of  Fourier  transforms.  Shinbrot's  work 
was  done  in  I960.  It  turns  out  that  the  concept  of  a  "properly  posed"  problem 
(^fo  ;  Hadamard;  page  33  J,  a  concept  of  the  first  importance  for  partial  differ¬ 
ential  equations,  was  introduced  by  Hadamard  in  1917.  In  1938  Petrovskii, 
motivated  by  the  above  mentioned  work  of  Hadamard,  used  a  Fourier  trans¬ 
form  to  investigate  the  proper  posing  (the  phrase  "correct  setting"  is  more 
common  in  the  Russian  literature)  of  the  Cauchy  problem  for  one  or  several 
partial  differential  equations  when  the  initial  functions  are  assumed  bounded 
(%.%  ;  Petrovskii  ^  .  We  observe  that  both  Shinbrot  and  Petrovskii  obtain  im¬ 
portant  and  similar  results  when  the  Fourier  transforms  increase,  in  the 
neighborhood  of  infinity,  no  faster  than  some  power  of  the  independent 


J 


Page  35 


variable.  Moreover,  in  1957  Gel'fond  and  Silov  generalized  the  work  of 
Petrovskii  by  constructing  the  Fourier  transform  for  functions  which  increase 
with  arbitrary  rapidity  (A  ;  Gel'fond  _)  .  It  is  entirely  possible  that  a  continu¬ 
ation  of  the  work  of  Shinbrot  in  analogy  to  that  of  Gel'fond  and  Silov  would 
produce  results  of  importance  for  integral  equations  of  the  first  kind. 


At  least  as  important  are  two  other  pertinent  concepts  of  Hadamard,  the 
notions  of  finite  part  (p.f. )  and  logarithmic  part  (p.l.  )  of  divergent  integrals, 
which  are  basic  to  the  "method  of  singularities"  developed  in  the  years  from 
1904  to  1932  as  an  alternative  approach  to  the  use  of  Fourier  methods.  Rather 
than  describe  these  concepts  in  general  forms  we  now  proceed  to  the  exposi¬ 
tion  we  have  outline  d,w  or  king  in  the  new  concepts  as  specific  questions  re¬ 
quire  their  introduction. 


The  results  in  Chapter  I  depend  on  showing  the  equivalence  of  the  two  integral 
equations  introduced  by  Youla,  namely. 


QJ 


* 


(1-1-5) 


t'C)  s.  f  gy  l* ,  ^(T)  ^  -fc-T  a'c  A  -tr 

-b-T 


K-r 


and  the  equations 


(1-2-2) 


f  'Sj.  LtyX)  o*(*)  dir.  ^  1  <\tt) j  t-T+xi* 

©a. 


■fc-T' 

tf 


j  'Si (V) ^,09  - As. 


-fc-T 


Since  we  are  given  the  covariance  functions  and  £  C*,s> 

we  first  show  that  (1-1-5)  implies  (1-2-2)  by  construction  of 
and  T?  .  It  is  then  easily  seen  how  to  reverse  the  arguments 

to  show  that  (1-2-2)  implies  (1-1-5).  Since  the  construction  of  the  kernels 
12l  LZ  and  Ex  Lbs)  follow  the  same  pattern  we  choose  to 

discuss  in  detail  only  one  of  them,  in  particular  »  since  the 

notation  in  its  equation  is  a  bit  simpler.  Since  the  covariance  function 
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is  continuous,  positive  definite,  and  symmetric  ;  Youlaj 

page  100  it  can  by  Mercer's  Theorem  be  expanded  into  the  absolutely  and 
uniformly  convergent  series 


(2-1-1)  (^js) 


t>Q 

1L 


A,  "  l 


j  t  -T  ^  SjS  4.  -t 


where  the  tpAXYl  and  are  the  eigen  functions  and  eigenvalues  of  the 

kernel  ,  i.e.,  they  satisfy 


(2-1-2) 


f 

15  m.  (_V)  ^i(s) 


b'T 


In  addition, the  4-U.(5)  are  orthonormal,  i.e., 


(2-1-3)  ^  ^15)  +■*  =  ^3 

-t-r 

and  because  of  the  positive  definite  nature  of  R  M.  foo 

^  is  complete  Cs‘*  >  Tricomi;  page  124^  . 


the  set 


The  general  analysis  utilizing  (2-1-1)  with  an  infinite  number  of  terms  is 
going  to  be  lengthy.  To  obtain  insight  into  the  construction  and  to  verify  in 
complete  detail  the  results  of  Chapter  I  in  an  at  least  restricted  setting  we 
first  restrict  (2-1-1)  to  a  finite  number  hi  of  terms  obtaining  what  is 
called  a  degenerate  or  Pincherle-Goursat  kernel  £ 3i  ;  Tricomij  page  55  3 
With  this  restriction  we  now  assert  that  the  kernel  Kx  (  is  given 

by 

NJ 

(2-1-4)  -  Tl  /U,*,  ^(i)  ^CS> 

I 

To  prove  this  we  need  only  insert  (2-1-4)  in  the  left  hand  side  of  the  second 
equation  in  (1-2-2)  and  show  that  the  right  hand  side  results.  We  obtain 
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(Z— 1  — f 


^  §  'Rj^  Cijs)  ^  e,  (.s)  -  ej.1 ^ft*ts>3  j  ■= 

-i-T 


^  $-(*><£*  ^  'Rx(^S>  <*-S 


-t-r  -t-r 


to 


Z 


A- 


<*(T)4vO)  cLT 

t-r 


xd)  =  j.«> 


t  he  critical  steps  in  the  manipulation  following  from 

(z  1  6)  ^  (?,s)  — 

-t-r 


-A.  -I 


where  K-x  (^5^  is  the  "unit  kernel"  with  the  property 

(2'1"7)  k z(tj)  =  <%Li) 

4  -r 
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which  is  just  the  last  equation  in  (2-1-5).  Note  that  the  manipulations  from 
(2-1-4)  to  (2-1-7)  are  critically  dependent  on  the  finite  N  assumption  which 
implicitly  carries  with  it  the  completeness  of  the  set  of  functions  J  > 

X.  |  j _ \J  .  The  removal  of  this  assumption  will  introduce  into  the 

manipulations  convergence  difficulties  relative  to  the  infinite  series  to  be 
considered  and  consequent  difficulty  in  interchange  of  integration  and  summa¬ 
tion  in  reasoning  as  in  (2-1-5).  The  class  of  functions  to  be  considered  must 
then  be  widened  from  the  continuous  functions  of  the  finite  dimensional  case 
to  the  L x  functions  and  even  beyond  in  some  cases  to  distributional  solutions 
which  are  also  called  generalized  functions.  Youla  was  not  unaware  of  these 
considerations.  See  in  particular  his  Appendix  A  (39  ;  Youla;  page  100  ^  . 
Before  plunging  into  these  matters  we  summarize  our  results. 

Theoremi  Restricting  the  positive  definite  kernels 
'fca. C 't)  and  R/w-Ob*)  to  a  finite  number 
N  of  eigen  functions  and  eigenvalues,  the  so-called 
degenerate  or  Pincherle-Goursat  kernels,  it  is  pos¬ 
sible  to  define  inverse  or  reciprocal  positive  definite 
kernels  and  so  that  the 

set  of  equations  (1-1-5)  is  equivalent  to  the  set  (1-2-2). 
Consequently  the  assumptions  in  Chapterl  that  invoked 
later  demonstrations  in  Chapterl!  are  validated  under' 
the  above  mentioned  restriction  to  N  dimensional  space. 


Proof;  Given  the  positive  definite  expansions 


(2-1-8) 

K >, 

=  1L 

A.  -I 

NJ 

f^cg) 

X*. 

(2-1-1) 

(.V) 

=  r 

A.  ~  I 

'■M*) 

A-  3 

define  the 

inverses 

hJ 

(2-1-9) 

R  j-  v 

-  T* 

\ 

Nj, 

'Xv.  tf/.C't;) 

(2-1-10) 

R  -L  (tjS) 

Av  x 

- 

\ 

A-  MW.V'W*) 

which  are  clearly  positive  definite.  By  direct  substitution  in  (1-1-5)  and 
(1-2-2)  and  manipulations  analogous  to  (2-1-4)  to  (2-1-7)  the  equivalence  of 
(1-1-5)  and  (1-2-2)  is  clear.  Moreover  the  symmetry  implicit  in  the  con¬ 
struction  of  and  (JJ)  shows  that  the  postulate  (1-2-5) 
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is  actually  not  a  postulate  but  a  consequence  and  hence  non- symmetric  kernels 
play  no  part  in  the  theory.  Q.  E.  D. 


To  investigate  the  general  case  we  streamline  our  notation  and  consider  the 
Fredholm  integral  equation  of  the  first  kind. 

(2-1-11)  <p(*>  =  V'  KLx,?))  Uty  ,  —I  4  I 

J-t 

where  we  assume  the  kernel  K-CXy*)  )  is  continuous,  symmetric,  and  posi¬ 
tive  definite  and  consequently  possesses  a  complete  orthonormal  set  of  eigen¬ 
functions  and  associated  set  of  eigenvalues  $  ^  • 

Results  applicable  to  Youla's  equations  will  be  collected  at  the  end  of  this 
section  and  exhibited  in  the  next.  To  further  simplify  manipulations  we  in¬ 
troduce  the  notation 

(2-1-12)  *' 

I 

-t 

so  that  the  eigenfunctions  and  eigenvalues  satisfy 


Note  that  capital  letters  denote  linear  operations  while  small  letters  with  an 
arrow  above  denote  functions  and  small  letters  without  the  arrow  are  scalars. 
This  is  the  usual  notation  in  studying  integral  equations  when  one  thinks  of  a 
function  as  a  vector  in  an  infinite  dimensional  space.  Even  without  this  atti¬ 
tude,  the  simplification  in  expression  is  worth  the  identification  as  the  follow¬ 
ing  illustrates.  We  rewrite  (2-1-11)  as 

(2-1-14)  ~  K  -f 

Our  objectives  in  this  section  are  centered  on  the  question  of  the  possibility 
of  writing  the  solution  of  (2-1-14)  as 

(2-1-15)  "f  =  f 
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The  class  of  functions  in  which  we  find  -f  clearly  depends  on  the  nature 
of  both  Ic'1  and  dj5"  .  From  (2-1-4)  we  can  immediately  obtain  the  Fourier 
components  of  -f  •>  relative  to  the  complete  orthonormal  set  of  functions 
t  it  >  by  taking  the  inner  product  of  (2-1-14)  with  to  Obtain 

(2-1-16)  ct  &.)  = 

where  we  have  used  the  symmetry  of  to  provide  the  critical 

manipulation  in  (2- 1  - 16),  i.e., 

(2-1-17)  (  |CT,  ?2.)  =  [tj  K&) 

From  (2-1-16)  we  have  the  desired  result 

(2-1-18)  (T,  ^ 


We  now  invoke  the  Riesz-Fisher  Theorem  [  L  ;  Courantj  page  110  ^  in  the 
formulation  due  to  Riesz. 

Theorem  (Riesz  1907):  If  \  is  an 

arbitrarily  given  complete  orthonormal  system  of 

functions  and  if^a, ,  «■!., -  are  arbitrary  real  numbers 

for  which  ’2^  converges,  then  there  ex¬ 

ists  a  summable  function  f-Cx')  with  summable  square 
for  which  <*-1  ■= 

By  virtue  of  the  above  theorem  there  are  only  two  possibilities:  either 
(I)  There  is  a  unique  (neglecting  functions  which  vanish  almost  everywhere) 
function  satisfying  (2-1-14)  which  can  be  calculated  as  the  limit  in  the 
mean 


oo 

;  or 


a.  N 

(2-1-19)  4-  -  ^  "Xa. 

oO  /C  =  | 

where 

(2-1-20)  'Zi  C)  < 


so  that  the  solution  -f  is  in  L*. 


(II)  The  infinite  series  diverges,  i.e,, 

(2-1-21)  ^  (f,?.)'2'  =  °° 

A.-I 

and  (2-1-14)  has  no  solution  of  class  ^  . 

To  appreciate  how  stringent  a  condition  (2-1-20)  imposes  on  T  we  now 
show  that  Xa.  -*■  ***  as  .  This  follows  from  the  fact  that 

(2-1-22)  fv  L'Xy'b)  -  X  ^C~ ^ 

where  by  Mercer's  Theorem  [(>  ;  Courant  Vol.  I;  page  138  )  the  convergence 

is  uniform  as  a  consequence  of  the  continuous,  symmetric,  positive  definite 
nature  of  .  From  (2-1-22)  we  find 

oo 

-c=<  "  'IS^I 

so  that  as  x. in  order  that  the  series  converge. 

Since  the  simultaneous  satisfaction  of  (2-1-14)  and  (2-1-15)  implies 


(2-1-24) 


(C  K  1  ~  K~'  K  -  X^ 


the  identity,  it  is  clear  that  the  eigenvalues  and  eigenfunctions  of  ft-  1 
follow  from  the  calculation 


(2-1-25) 


•=  {c  '  fc  <p^  -  X  c^.  -  ’eft 


so  that 


(2-1-26)  <fL  -  j  l>X> - __  , 

i.e.  ,  the  inverse  kernel  has  the  same  eigenfunctions  but  inverse  eigenvalues. 
The  impossibility  of  the  simultaneous  satisfaction  of 


(2-1-27)  X  Xr  ^  ^  ^  ^ 

A'”  /  ^ 

which  would  follow  if  we  tried  to  express  K;  C'VSj)  in  analogy  to 
(2-1-22)  shows  us  the  necessity  of  adopting  a  fresh. approach  in  representing 
K~'  . 


C  &o 
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What  we  have  hinted  at  in  the  above  observations,  in  particular  in  noting  that 
— ip,  ©£>  cl  s  ji  — ^  <*»  and  that 

(2-1-18)  f-)  ~  j  j 

(2-1-26)  k-'  -=  'X-c  <$l  ,  b  x.>  -- -j 

is  that  both  the  solution  and  the  operator  k_l  may  be  unbounded  in 
general  and  require  expression  through  the  concept  of  distributions  or 
"ideal  functions".  The  name  "distributions"  indicates  that  ideal  functions, 
such  as  Dirac's  delta -function  and  its  derivatives,  may  be  interpreted  by 
mass  distributions,  dipole  distributions,  etc.,  concentrated  in  points,  or 
along  lines,  or  on  surfaces,  etc.  An  excellent  introduction  to  ideal  functions 
or  distributions  is  available  in  Courant  (_  7  S  Courant  Vol.  II;  pages  766-  798) 
but  in  the  following  pages  we  choose  to  give  meaning  to  the  phrase,  "Either 
the  kernel  is  an  ordinary  function  or  it  must  be  expressed  as  a  distribution, 
that  is,  obtained  from  continuous  functions  by  differentiation  processes" 

(_?  ;  Courant  Vol.  II j  page  727)  .  On  the  same  page  in  a  footnote  Courant 
continues  '  "Such  a  representation  for  a  single  equation  of  second  order  is 
the  subject  of  Hadamard'sfamous  theory.  " 

What  we  attempt  to  do  in  the  following  is  to  give  an  explicit  example  of  how 
Hadamard,  seeking  an  integral  representation  of  the  Cauchy  problem  for  the 
wave  equation  (to  be  defined)  had  to  generalize  the  meaning  of  integral  to 
achieve  the  representation.  In  the  words  of  Courant,  "Hadamard's  invention 
of  the  finite  part  (of  a  divergent  integral)  may  be  regarded  as  an  important 
motivation  for  the  modern  theory  of  distribution"  (_7  ;  Courant  Vol.  II;  page 
743)  .  In  view  of  our  interest  in  integral  representations  in  Section  I  the 
possible  importance  of  the  above  concepts  to  our  theory  should  be  evident 
to  the  reader. 


The  following  formula  are  in  the  notation  employed  by  Bureau  in  a  paper 
published  in  1955 C-^ ;  Bureau;  page  1 54)  .  We  give  only  enough  of  the  prob¬ 
lem  to  indicate  the  form  of  the  difficulty  faced  by  Hadamard  and  the  final  form 
of  solution  achieved.  Detailed  study  of  Bureau  to  appreciate  the  constructions, 
by  no  means  trivial,  is  recommended  to  the  reader.  That  the  notion  of  a 
finite  part  of  a  divergent  integral  seems  at  first  sight  somewhat  strange  was 
commented  on  in  "An  Essay  on  the  Psychology  of  Invention  in  the  Mathematical 
Field"  where  J.  Hadamard  himself  wrote:  "Certainly,  considering  it  in 
itself,  it  looks  typically  like  'thinking  aside'.  But  in  fact,  for  a  long  while 
my  mind  refused  to  conceive  that  idea  until  positively  compelled  to.  I  was 
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led  to  it  step  by  step -  I  could  not  avoid  it  any  more  than  the  prisoner  in 

Poe's  tale  The  Pit  and  the  Pendulum  could  avoid  the  hole  at  the  center  of 
his  cell",  (j?.  i  Hadamard;  page  110). 

We  consider  real  quantities  ICi, - ,  y-y ,  u.  and  write  as  abbreviated 

notations  (_  S'  ;  Bureau?  page  154) 


(2-1-28) 


Y-  —  (-  - >  y-f)  *9”  ( *0 1  >  -  -  >  3-  f )  ) 

Zc(X)-t)  ^  it  (.  vt> - j  A-1  -  tA>o) 

Au  -  4. _ + 

a  a  y.p" 


We  consider  the  Cauchy  problem  for  the  wave  equation,  i.e.  ,  consider 


(2-1-29) 


—  A  U- 

a-b'*' 

-=■  O  j  lA-fc(_x;o)  - 


where  is  a  regular  function  and  we  put  *«- C °)  -  0  to  obtain 

simple  formulas.  The  case  for  It CX;o}  o  can  easily  be  solved  once 
the  above  problem  is  resolved  (_5~  ;  Bureau;  page  155).  We  shall  say 
that  a  function  <^(>)  is  regular  if  it  is  continuous  together  with  its  deriva¬ 
tives  up  to  a  certain  order  s  ;  the  order  will  vary  according  to  the  nature 
of  the  problem.  This  class  of  functions  occurs  over  and  over  again  in  the 
work  of  Hadamard  and  we  will  meet  it  later  in  discussing  his  concept  of 
"properly  posed"  problems.  The  solution  of  (2-1-29)  is  given  by  (S'  j  Bureau; 
page  157) 

j>-z. 

(2-1-30)  lc(x)t)  -  -J—  ^ - -  \  %(«»<>  cLoC 

(B-T)!  3-tT*'1-  ) 

where 

(2-1-31)  (.a)  x=  — - —  \  ^  -0-A 

-XX.  /v  0  Ck.  Vj  n. 


is  the  mean  value  of  the  regular  function  on  the  hypersphere  JXXjt, 

of  center  X  and  radius  A  in  the  j>  — dimensional  Euclidean  space.  The 
temptation  is  great  to  rid  (2-1-30)  of  the  derivatives  by  differentiating  the 
integral  but  since 


Lr_Z  4 

2- 


O 
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the  resulting  terms  in  C-fc1- would  be  raised  to  negative  powers  and 
consequently  diverge  for  oi-=  ,  i.e.  ,  at  the  end  of  the  interval.  Yet  this 
is  the  key  to  the  problem.  In  order  to  obtain  an  "integral"  from  (2-1-30) 
one  must  generalize  the  meaning  of  integral  to  cover  the  case  where  infinities 
occur.  Hadamard  conceived  the  idea  of  discarding  the  "infinite  part"  of  the 
integral  and  saving  the  "finite  part"  (p.f.  )  and  moreover  was  able  to  find  a 
consistent  setting  for  his  concepts.  Little  wonder  that  he  felt  drawn  to  the 
"pit"  1  He  thus  achieved  one  of  the  very  basic  advances  in  mathematics,  the 
extension  of  a  given  set  of  mathematical  objects  by  additional  new  "ideal 
elements  "not  defined  as  entities  in  the  original  set  %  (here  not  defined  be¬ 
cause  of  divergence),  and  not  defined  descriptively  but  defined  merely  by 
relationships  such  that  in  the  extended  set  ~i&  the  original  rules  for  basic 
operations  are  preserved. 

The  final  form  of  Hadamard's  results  is  sensitive  to  the  even  or  odd  nature 
of  p,  which  is  tied  up  to  the  study  of  propagation  phenomena  and  Huyghens' 
Principle  valid  only  for  p  odd  and  greater  than  one(  7  5  Courant  Vol.  II }  page 
764  ),  so  that  separate  new  concepts,  finite  part  (p.f.  )  and  logarithmic  part 
(p.l. )  of  divergent  integrals  enter  the  followings 

For  p  even,  (2-1-30  )  can  be  replaced  by 

(2-1-32) 

where 
(2-1-33) 

For  p  odd,  (2-1-30)  can  be  replaced  by 

(2-1-34)  let*}-*)®  ^  {.*>  -A?)  ^ 

where 

(2-1-35)  l>f  -  C-0  ^  TT  n(_  ^zr) 

[  — p 

Since  (-fcv—  /i*)  is  a  solution  of  Tx-e-t  —  =  o  the 

reader  should  note  the  resemblance  of  (2-1-32)  and  (2-1-34)  to  the  usual 
convolution  formulas  used  in  communication  theory.  For  precise  definitions 
of  finite  part  (p.f. )  and  logarithmic  part  (p.l. )  of  the  divergent  integrals  see 
Bureau(  S'  ;  Bureau;  page  146  )  .  The  number  of  derivatives  s  of 


lei*}*)  =  Af  j 

*  1 

At  -  (-0  ^  a-'  r(HL) 
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that  enter  (2-1-32)  and  (2-1-34)  are  then  clearly  seen  and  it  follows  that  the 
solution  depends  in  a  continuous  fashion  on  (to)  and  derivatives 

of  V*1)  up  to  or^er  s  ’ 


The  nature  of  the  above  solution  contrasts  sharply  with  the  behavior  of  the 
solution  of  the  Cauchy  problem  for  Laplace's  equation,  i.  e.  ,  consider 


(2-1-36) 


_  T-  _  Z. 

c>  tt  ,  3  it  0 

*  a*  ~ 

ZtC*j0)  ~  $iOo>  "h-tO-jo)  *=. 


In  his  Yale  lectures  (j°  ;  Hadamard;  page  Xz  _)  Hadanoard  made  the  follow¬ 
ing  observation  concerning  the  solution  of  (2-1-36).  It  is  possible  to  modify 
the  solution  of  (2-1-36)  arbitrarily  much  and  arbitrarily  near  the  line  -t  =  o 
by  changing  the  values  of  and  -x.  tx)  and  of  all  their 

derivatives  at  -t  •=■  o  as  little  as  we  please.  To  do  this  one  adds  to 

the  given  solution 

—  ^  M?  , 

(2-1-37)  ~  2-  /w  Y  /Wb 


where  /w  is  a  positive  constant.  The  function  aK,  which  is  a  solution  of 
the  differential  equation  in  (2-1-36),  and  an  arbitrary  number  of  its  partial 
derivatives  with  respect  to  "b  and  y.  are  arbitrarily  small  at  "t  — »  ,  if 
aa-  is  chosen  sufficiently  large,  but  for  ■>  t>o  the  function 
assumes  arbitrarily  large  values  dependent  on  the  choice  of  av,  . 


Hadamard  then  draws  a  distinction  between  "properly  posed"  problems,  of 
which  (2-1-29)  is  an  example,  and  "improperly  posed"  problems,  of  which 
(2-1-36)  is  an  example.  More  precisely  we  say  that  a  Cauchy  initial  value 
problem  is  properly  posed  if  there  exists  a  number  5  such  that  for  initial 
functions  that  are  regular  of  order  3  ,  i.e.  ,  continuous  to¬ 

gether  with  their  derivatives  up  to  order  S  ,  then  there  exists  exactly  one 
solution  and  Xc Cx;*)  depends  continuously  on  the 

and  on  their  derivatives  of  order  S 

In  1938,  I.  G.  Petrovskii  (it  ;  Petrovskii  3  motivated  by  Hadamard's  ideas 
analyzed  by  Fourier  methods  the  following  system  of  linear  partial  differ¬ 
ential  equations 

(2-1-38)  Lbdss.  =  ntsSa 

where  -  ^U,L-np^>) - }  'U**.  (x/t-j  j  is  the  desired  vector 
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function  with  /»v  components,  depending  on  the  time  ir  and  also  on  the 
spatial  point  -x  -  (  v  i  ,  -  —  ;>  x  m  )>  .  T!  is  a  matrix  with  /yv.  rows 

and  /w  columns,  whose  elements  are  polynomials  in  the  differential  opera¬ 
tors  with  respect  to  the  variables  j  -  'j> - Kl  of 

various  orders  and  with  coefficients  which  depend  continuously  upon  the  time. 
The  initial  conditions  are  given  by 

(2-1-39)  -  V-o(x) 


where  Ho  (Jo)  is  a  given  vector  function.  Applying  a  Fourier  transform, 
denoted  by  >  -  ttC*/*-.)  to  (2-1-38)  we  obtain  the  system  of 

ordinary  differential  equations, 


(2-1-40)  <L  aT  tu,*)  _  ^(trfytr)  . 

where  the  matrix  E  (»<>■<=  >  is  obtained  by  the  correspondence 


1-41) 


oC 


=  k  — .  * 


Petrovskii  obtains  a  necessary  and  sufficient  condition  that  (2-1-38),  (2-1-39) 
be  properly  posed.  (Actually  Petrovskii's  setting  is  slightly  more  general, 
the  precise  concept  he  investigated  being  that  of  "uniformly  properly  posed" 
problems  where  the  initial  instant  is  not  necessarily  "t-o  Ut  o  , 

but  we  shall  not  introduce  the  additional  notation  here^xz-  ;  Petrovskii;  page 
3  .  )  The  distinguished  condition,  called  Condition  A  in  the  literature,  ex¬ 

presses  a  restriction  on  a  fundamental  set  of  solutions  of  (2-1-40)  relative  to 
their  rate  of  growth  with  respect  to  ,  where 

(2-1-42)  o4  |*|  —  rvYu>-?c  (o<k|  j  I j - j  ^  ) 

as  -V  »o  .  In  essence  Condition  A  states  that  no  solution  of  (2-1-40) 
grows  faster  than  a  fixed  power  of  as  oi ^  ^  o»  .  The  point  of  intro¬ 

ducing  fundamental  solutions  is  that  the  above  condition  need  be  checked  only 
for  <y\.  independent  solutions  of  (2-1^40).  The  number  S  in  the  definition  of 
properly  posed  can  be  related  to  the  maximum  power  of  in  the  verifi¬ 

cation  of  Condition  A,  when  the  condition  is  fulfilled.  This  fact  is  our  con¬ 
necting  link  with  the  work  of  Shinbrot  on  integral  equations  of  the  first  kind 
and  ultimately  with  Youla's  equations! 


In  I960  Shinbrot(.:W;  ShinbrotJ)  studied  the  integral  equation  (2-1-11)  with  the 
additional  restriction 
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(2-1-43) 

i . e .  ,  that 
becomes 

(2-1-44) 


is  a  difference  kernel.  The  equation  (2-1-11)  then 
r  I 

-  \  lc(  £(-$)  ^ 

-I 


Using  the  same  notation  as  with  Petrovskii  but  noting  that  now  X  and  c< 
are  one  dimensional  we  denote  the  Fourier  transform  of  ixi^> 

In  order  to  state  his  results  Shinbrot  makes  three  assumptions  ;  Shinbrot; 
page  4  ) 


(i)  The  derivative 

'doL 

is  continuous  for  real  and  belongs  to  U-^ 

(ii)  The  derivative 


as  jot  (  — ^  along  the  real  axis . 

(iii)  The  quantities 


fC  (.■*)  — 


K'U) 


h'  ~ 


are  both  finite. 


Note  that  for  fc(  IX  —  ^  a  covariance  function  it  follows  that  HkT(°c) 

is  real  so  (ii)  is  trivially  satisfied.  The  assumption  (i)  is  familiar  from 
consideration  of  network  synthesis  ideas  and  says  essentially  that  an  accept¬ 
able  does  not  change  too  rapidly.  Recalling  Condition  A  from  the 
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work  of  Petrovskii,  "No  solution  of  (2-1-40)  grows  faster  than  a  fixed  power, 
say  2  ,  of  ck-s  oC  — ►  "we  see  that  assigning  £"(<*.>  the 

rate  of  growth  from  Condition  A  yields  in  (iii) 

(2-1-45)  X  =  U  - 

<X_ 

We  are  not  of  course  asserting  a  direct  connection  but  merely  that  the  rate  of 
growth  of  Petrovskii's  solutions  yield  finite,  acceptable  limits  in  (iii). 

At  this  point  we  quote  pertinent  sentences  from  Shinbrot  concerning  this  pur¬ 
poses  and  his  main  result  which  we  summarize  as  a  Theorem  (^-c  ;  Shinbrot; 
page  5).  "In  most  applications,  one  is  interested  in  solutions  that  belong  to 
some  U  —  class.  In  the  case  of  the  problem  of  a  filter  with  a  finite  memory, 
however,  the  solution  itself  represents  an  operator  that  is  to  be  applied  to  the 
signal  plus  the  noise  to  obtain  the  best  linear  approximation  to  the  signal  it¬ 
self.  Consequently  the  solutions  of  interest  do  not  necessarily  have  to  belong 
to  any  L  -  class;  indeed  they  need  not  be  functions  at  all  but  may  be  dis¬ 
tributions  in  the  sense  of  Laurent  Schwarz  (e.g.  ,  functions  and  their  de¬ 
rivatives)"  . 

Theorem:  (Shinbrot)  If  is  used  to  denote  the 

integral  part  of  the  solution  of  (2-1-44)  consists 

of  an  "ordinary"  function  belonging  to  some  L-class  plus 
a  finite  sum  of  derivatives  of  the  %  -function,  the  highest 
derivative  that  can  occur  being  I  . 

Remark:  Of  the  many  possible  approaches  to  the  ^  -function  and  its  deriva¬ 

tives  currently  available  Shinbrot  depends  on  the  treatment  of  Lighthill  which 
is  especially  suited  to  Fourier  methods  £  7  ;  Lighthill  p-  ICl}. 

Example :  For  k^lXl)  =  e.  *Vl  )  X  -  ?L  so  a  tx)  can  occur  in 
the  solution  of  (2-1-44). 


2.2  APPLICATION  TO  YOULA'S  EQUATIONS 

For  finite  dimensional  kernels  the  theory  is  complete  as  indicated  in  the 
first  theorem  in  the  last  section.  The  general  situation  when  one  attempts  to 
produce  kernels  Ri(S;t)  and  Ri  to  show  the  equiva¬ 

lence  of  Youla's  equations  (1-1-5)  and  the  equations  (1-2-2),  which  are 
solved  for  %(*)  in  order  to  eliminate  and  produce  the  functional 
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x  kaus  ,.  is  that  distributional  solutions  are  appropriate.  This  is 
especially  clear  from  (2-1-18)  and  (2-1-26)  which  indicate  in  general  notation 
that  both  the  function  -f  and  the  operator  K_1  associated  with  (2-1-15)  may  be 
unbounded.  That  integral  operators  are  bounded  and  differential  operators 
are  unbounded  is  well  known.  The  question  then  arises  j  must  differential 
operators  be  admitted  to  the  theory  and, if  so,to  what  extent? 

At  this  point  we  attempted  to  illustrate  two  concepts  of  J.  Hadamard  that  are 
of  great  potential  for  further  progress  in  analyzing  Youla's  equation.  The 
first  was  the  concept  of  finite  or  logarithmic  part  of  a  divergent  integral, 
the  divergency  occuring  when  one  attempted  to  eliminate  the  derivatives  ap¬ 
pearing  in  (2-1-30), the  solution  of  (2-1-29).  The  use  of  divergent  integrals 
in  the  systematic  fashion  initiated  by  Hadamard  and  carried  on  today  by 
Bureau  is  an  approach  almost  unknown  in  communication  theory.  One  of  the 
main  virtues  of  the  approach  is  the  directness  with  which  it  answers  the 
question  raised  above  concerning  the  number  of  derivatives  that  must  appear 
as  a  result  of  the  generalized  meaning  of  the  integral.  See  (2-1-32)  to 
(2-1-35)  and  the  discussion  that. followed. 

The  second  and  related  concept  was  the  properly  posed  nature  of  a  problem 
which  implied  through  subsequent  work  of  Patrovskii  an  approach  through 
Fourier  integrals  that  again  emphasizes  how  many  derivatives  must  enter 
the  solution  of  a  Cauchy  problem.  The  similarity  between  this  work  and 
that  of  Shinbrot  which  is  applicable  for  the  specialization  to  difference  ker¬ 
nels,  hence  stationary  statistics,  seems  especially  strong  in  light  of  equation 
(2-1-45).  Finally  the  generalization  of  Petrovskii's  work  by  Gel'fand  and 
Silov^  ;  Gel'fand  )  may  provide  tools  to  generalize  Shinbrot  and  ultimately 
Youla's  work. 
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CHAPTER  III 

COMMUNICATION.  SYSTEMS  BASED 
ON  HADAMARD  MATRICES 


3.  1  INTRODUCTION 

In  1909  Hadamard's  Theorem  on  the  maximum  value  of  a  determinant  with 
bounded  elements  was  utilized  to  prove  the  convergence  of  the  celebrated 
Fredholm  Formulae  ;  Tricomi;  page  66  )  which  explicity  represent  the 
resolvent  kernel  of  an  integral  equation  of  the  second  kind  with  a  bounded 
kernel.  Having  spent  the  first  two  chapters  reasoning  with  integral  equations 
of  the  first  kind  it  is  pertinent  to  remark  that  when  the  kernel  in  an  equation 
of  the  first  kind  is  a  composite  of  an  ordinary  function  and  a  distribution  or 
ideal  function  then  the  equation  of  the  first  kind  includes  within  its  scope 
equations  of  the  second  kind  and  integro-differential  equations.  The  central 
idea  of  Fredholm's  method  was  a  heuristic  construction  for  a  bounded  kernel 
KCX,"^)  that  would  mimic  the  finite  dimensional  construction  applicable 
for  a  degenerate  or  Pincherle- Goursat  kernel  .  Rather  than 

attempt  to  justify  all  aspects  of  the  passage  to  the  limit  as  v*.  ,  which 

would  correspond  to  the  approach  of  the  last  chapter,Fredholm  simply  wrote 
down  the  infinite  series  forms  made  plausible  by  finite  dimensional  arguments 
and  directly  verified  their  convergence  by  means  of  a  majorant  series,  the 
convergence  of  the  majorant  series  following  from  Hadamard's  Theorem.  We 
now  give  an  explicit  statement  of  the  theorem  ( 31/  ;  Tricomi;  page  223^)  , 


Theorem  (Hadamard)  If  D  is  a  determinant  of  the  n-th 

order  of  the  matrix  \  \  ,  (/l,  S  -  I > - ,  •*-  3  , 

then 

(3-i-ij  *  t i 

S-l  S-»  S-I 


where 


TT  |( 

a -i 


(3-1-2) 


<x.v  -  C  Q-ai  J  > - )> 

W. 

K’tX \\u  - 


Moreover,  equality  in  (3-1-1)  is  attained  only  when  the  row  vectors 


are 
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i.  e .  , 

/w 

■s.  2>  P 

S  =1 

Geometrically  (3-1-3)  implies  that  the  volume  of  the  polyhedron  formed  from 
/*-  vectors  of  given  length  in  /w^  dimensional  space  is  greatest  if  the  factors 
are  mutually  orthogonal.  Provided  that 

(3-1-4)  Kas  |  f  B 

it  follows  from  (3-1-1)  that 

-J  M.  A'- 

(3-1-5)  ~J)  ^  /v^  "B 


mutually  orthogonal, 
(3-1-3) 


Modern  communication  theory  assumes  a  normalization  of  B,  namely  B=l, 
so  that  the  following  definition  emerges (^-1  ;  P'etersonj  page  79).  A  Hadamard 
matrix  H  is  an  orthogonal  matrix  whose  elements  are  the  real  numbers 

+1  and  -1.  The  orthogonality  is  in  the  sense  of  (3-1-3). 

Our  purpose  in  this  section  is  to  present  some  communication  system  tech¬ 
niques  that  are  derivable  from  a  mathematical  concept  that  reduces  to  a 
Hadamard  matrix  in  certain  special  cases  and  that  constitutes  a  generalized 
Hadamard  matrix  in  the  remaining  cases.  The  sense  of  the  generalization  is 
relative  to  the  concept  of  orthogonality.  Clearly  a  Hadamard  matrix  H  satis¬ 
fies  the  equation 

(3-1-6)  H  H  r  ^  ^  X 

where  T  means  transpose  and  I  is  the  identity  matrix. 

Any  matrix  satisfies  the  equation 

(3-1-7)  A  Ar  ^  -h'fc 

where  k  is  a  scalar  and  R  is  a  "remainder"  matrix.  We  associate  the  rank 
r  of  R  with  the  degree  of  orthogonality  of  A.  If  A;  O  ,  i.e.  ,  R  vanishes, 
then  A  is  an  orthogonal  matrix.  If  r  =  1,  we  say  that  A  is  "almost  orthogonal". 

The  cases  r=2,  3, - ,  n  correspond  to  lesser  degrees  of  orthogonality  and 

will  not  be  distinguished  by  name.  The  "almost  orthogonal"  matrices  consti- 
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tute  the  basis  of  a  multiplexing  technique  which  has  certain  attractive  features 
for  use  as  an  anti-jamming  communication  system.  The  basic  philosophy  of 
our  approach  is  that  orthogonal  matrices  are  of  too  simple  a  structure  to 
analyze  and  hence  jam,  but  that  almost  orthogonal  structures  are  sufficiently 
simple  to  construct  and  sufficiently  variable  to  make  jamming  difficult.  The 
multiplexing  approach  will  occupy  our  attention  in  section  3.2. 

Having  outlined  our  purpose  we  now  show  the  naturalness  of  our  approach  by 
putting  the  Hadamard  matrix  H  in  perspective  with  modern  coding  theory. 


It  is  important  in  the  theory  of  symmetric  binary  codes  Slepian  3  to 
determine  the  maximum  number  of  different  rw—  place  binary  se¬ 

quences  that  can  be  constructed  such  that  the  Hamming  distance  j  Hamming 
between  any  two  of  them  is  greater  than  or  equal  to  a  preassigned  positive 
integer  d  .  A  set  of  aw  av  -place  binary  sequences  such  that  the  distance 
between  any  two  is  greater  than  or  equal  to  cl  is  called  an  /w  -place  code  of 
length  />*_  and  minimum  distance  d  .  Denote  such  a  code  by  M  ( *> 

{y>;  PlotkinJ)  . 


Since  aw.  mutually  orthonormal  vectors  in  an  A"-—  dimensional  space  have, 
in  the  usual  Euclidean  metric,  the  maximum  possible  separation  one  suspects 
that,  mapping  the  real  numbers  +  I  and  —  I  into  the  binary  notation  of 
coding  theory,  a  Hadamard  matrix  can  be  associated  with  a  code  M(~j  <1 
That  such  is  indeed  the  case  is  the  content  of  the  following  theorem/a);  Peter¬ 
son;  page  79  )  . 

Theorem:  If  there  exists  „an  /w.JC/w,  Hadamard  matrix, 
there  exists  a  binary  code  f'l  where  aw-  Hw^  . 


Proof:  Let  H  be  a  Hadamard  matrix.  The  code  is  constructed  as  follows: 

Form  a  set  of  3l<v^.  vectors  a,  ,  - j - > 

where  the  a,  — _ are  the  rows  of  H  .  Then  in  each  of 

these  change  the  -1.V,  to  o1^  and  the  -lV  to  •  This  gives  a  set  of 

aw  •=.  3-w  vectors  of  /w  binary  symbols  each.  Since  corresponding  com¬ 
ponents  of  CL*.  and  -  <Vt  are  different,  the  Hamming  distance  between  al 
and  —  ou  is  /vo  >  Since  L  and  t  are  orthogonal,  equa¬ 
tion  (3-1-  3),  if  -j  ,  they  must  match  in  half  the  positions  and  differ 

in  the  other  half,  and  thus  the  corresponding  binary  vectors  are  at  distance 
■  Q.  E.  D. 


For  example,  it  is  easy  to  construct  a  Hadamard  matrix  H~  for  <w  =  o. 
where  K.  is  a  positive  integer.  The  resulting  code  is  the  same  as  a  first 
order  Reed-Muller  code(a*<  •  Reed  )  or  a  MacDonald  code  ( 1^  ;  MacDonald^)  . 
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That  codes  generated  by  Hadamard  matrices  are  maximal  in  the  sense  that 
no  other  code  with  the  same  minimum  distance  and  the  same  number  of  places 
has  greater  length  was  demonstrated  by  Bose  and  Shrikhande:  "This  note  es¬ 
tablishes  a  connection  between  Hadamard  matrices  H**  and  the  maximal  binary 
codes  M  (  4 -b > 'S-'fc  jtl?)  j  M  ( 4  *-  *  ;>  9c6  ^  4  +  3  M  C  4  +  j  •X-t) 

in  two  symbols  0  and  1,  where  by  M  ^  <m. )  ,  we  mean  a  set  of  tnn. 

n-place  sequences  with  0  and  1  such  that  the  Hamming  distance  between  any 
two  sequences  is  greater  than  or  equal  to  d"(«f  ;  Bose;  page  183^). 

In  order  to  state  more  explicitly  the  "connection"  in  the  above  quotation,  we 
now  introduce  the  concept  of  a  symmetric  balanced  incomplete  block  design, 
the  so-called  tS’>  X-  problem,  a  concept  sufficiently  general  to  include 
the  Hadamard  matrices  as  a  special  case. 

Foregoing  the  symmetric  case  for  the  moment,  we  define  a  balanced  incom- 
lete  block  design  as  an  arrangement  of  /O'  objects  into  b  sets  called  blocks 
such  that  every  object  occurs  exactly  fl  times  and  every  pair  of  objects 
occurs  exactly  \  times  in  a  block.  The  subject  of  the  design  of  experiments, 
in  which  the  above  concept  has  played  a  central  role,  was  built  up  largely  by 
two  men,  R.  A.  Fisher  a.nd  F.  Yates.  The  subject  of  the  enumeration  of 
designs  was  dealt  with  by  Bose  in  1939  whose  principal  tools  were  the  use  of 
finite  geometries  and  symmetrically  repeated  differences  (3  ;  Bose  )  . 

Viewing  the  design  of  experiments  as  an  attempt  to  extract  information  against 
a  background  of  interference  we  see  the  naturalness  of  Bose  producing  papers 
in  1939  and  1959  using  virtually  the  same  tools  on  what  at  first  sight  seem 
distinct  disciplines,  namely  design  of  experiments  and  code  construction. 

For  a  symmetric  design,  b=X"  and  ,  and  the  arrangement  is  called  a 

\  configuration.  Irrespective  of  its  connection  with  statistical  con¬ 
siderations,  as  outlined  above,  the  fS’j  {t,  X  problem  has  played  a  dis¬ 
tinguished  role  in  combinatorial  mathematics  (is';  Ryser;  page  456  J  . 

The  X  Problem.  Let  <"3"  elements  'K,  - - ^  X  >*•  be  arranged 

into  rr  sets  _ ,  X*-  such  that  every  set  contains  exactly  Jt  dis¬ 

tinct  elements  and  such  that  every  pair  of  sets  has  exactly  X-  elements  in 
common  (  ocX<  .  Such  a  configuration  is  called  a  X  con¬ 
figuration.  Let  -  1  if  is  in  set  and  »  otherwise. 

The  matrix  K  ~  of  order  /S'  is  called  the  incidence  matrix 

of  the  configuration.  Let  be  integers  such  that  . 

Then  a  configuration  exists  if  and  only  if  there  exists  a  (°> 

matrix  A.  of  order  /S'  such  that 

A  AT  -  'B 


(3-1-8) 
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where  AT  denotes  the  transpose  of  K  and  where  B  has  M-  in  the  main 
diagonal  and  X  in  all  other  positions.  Clearly,  the  4t,  on  the  main  diagonal 
of  $  is  associated  with  At,  ones  in  each  row  of  A  ,  and  X  elsewhere  is 
associated  with  the  overlap  of  X-  ones  in  each  distinct  pair  of  rows  of  A  . 
Since  each  element  occurs  equally  frequently  with  a  specified  one  within  a  seit  , 
we  have  that  AtO k-  i)  tz  X  i  so  that 

(3-1-9)  X  -  ‘fc-t'k-O 

A-  1 

The  central  problem  in  the  study  of  these  configurations  is  the  determination 
of  the  precise  range  of  values  of  .  i vT,  X  for  which  configurations  exist. 

Our  need  for  a  variety  of  configurations  is  dependent  on  specific  requirements 
of  the  multiplexing  scheme  we  have  in  mind?so  we  defer  the  general  question 
of  construction  until  later. 

A  specialization  of  <2.-,  X  to  /or  =  4^-1  ;  ,  X  -  *;  -  /  leads 

us  back  to  a  Hadamard  matrix  of  order  Todd^)  .  Moire  precisely,  the 

existence  of  such  a  A^k.>X  configuration  is  equivalent  to  the  existence  of  a 
Hadamard  matrix  of  order  ‘i'f  .  An  easy  construction  to  exhibit  the  equiva¬ 
lence  is  to  start  with  a  Hadamard  matrix  of  order  in  normal  form,  i.  e.  , 

by  permutations  normalize  the  Hadamard  matrix  to  have  only  -f  l's  in  the 
first  row  and  column,  delete  the  first  row  and  column  to  form  a  matrix  of 
order  aT" =  H~t-  I  >  and  change  the  -l's  to  0's.  Concerning  l's  there  are 
clearly  -fe.  s  a.-fc'-l  in  each  row  and  since  the  orthogonality  of  H  nt  required 
the  coincidence  of  l's  in  "t  columns  the  coincidences  have  been  reduced  to 
A  -  -b-l  .  Hence  the  resulting  matrix  meets  the  requirement  (3-1-8)  with 
tsr-  q  -k-t  ,  ,  X  ■=-  •fc-  t  and  the  reasoning  can  clearly  be 

reversed  so  the  equivalence  is  complete. 

With  the  above  concepts  in  mind  we  return  to  a  definitive  statement  of  Bose 
and  Shrikhande  in  the  following  theorem^;  Bose;  page  186). 

Theorem:  The  following  statements  are  equivalent: 

(a)  A  (q-t,2-fc)  yt  i.  e.  ,  there  exists  a  code  H,  (  4 cut  ^  Tfr  ) 

(b)  A  (f-fc-i  i.  e.  ,  there  exists  a  code  4'f) 

(c)  There  exists  a  configuration  with  the  following 

parameters , 

9  k'l  X-t-i  > 

(d)  There  exists  a  Hadamard  matrix  H4*  . 
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We  now  complete  the  cycle  of  reasoning  of  Bose  and  Shrikhande  by  stating 
a  result  of  Plotkin  ;  Plotkin)  . 

Theorem:  For  any  positive  integer  "fc 

(i)  k  a-t)  t 

(ii)  K  (4-t-  !_>**)  P  Hir 

(iii)  k  ^  ilP 

Combining  (a),  (b),  (c),  (d),  (i),  (ii)  the  maximal  nature  of  the  codes 

and  q-tr)  is  established. 

The  results  of  our  discussion  so  far  can  now  be  summarized  in  the  follow¬ 
ing  figure. 


FIGURE  4 

EQUIVALENCE  THEOREM  OF  BOSE  SHRIKHANDE 


The  most  notable  feature  of  the  research  summarized  in  Figure  4  is  the 
use  of  a  special  case  of  the  configurations.  We  set  ourselves  the 

task  of  obtaining  interesting  consequences  of  the  general  configura¬ 

tions  characterized  by  (3-1-8)  and  (3-1-9). 


We  first  rewrite  (3-1-8)  as 


(3-1-10) 
whe  re  T 


k  AT-  (4- A)  x  P 

denotes  the  identity  matrix, 


X  £ 

4 


denotes  a  matrix  of  all  l's  and 
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4-A 


>  O 


since 


O  <  A  <■  4  ^  ^ 


Now  observe  that 
(3-1-11)  A  £  ^  k-  & 

since  A  has  4  l's  in  each  row,  so  that  each  column  o£  is  an  eigen¬ 

function  of  A  with  eigenvalue  4  .  Note  that  ]£  is  of  rank  1  so  that 
A  is  "almost"  orthogonal  in  the  sense  of  the  discussion  following  (3-1-7). 
Putting  (3-1-11)  in  (3-1-10)  we  get 


(3-1-12)  A  A 


-  (4.- A)  j_  +-  A  A  ^ 
4 


so  that 


(3-1-13) 
which  yields 
(3-1-14) 

In  addition,  if  A 
(3-1-15)  l\~ 


A  UT-  %*  )  ’  (fc-A>X 


A'1  = 


-4  AT-  A  X 


<4  L'k-x) 

is  symmetric,  (3-1-14)  becomes 

—  -4  A  —  A  £  /\ 


4(4 -X) 

If  in  (3-1-15)  it  is  possible  to  choose 


(3-1-16) 


-I 


4= 

aJ“=  i#X-I. 


“AA 


=  A 

,  then  using  (3-1-9)  we  obtain 

Ar=  A 


Since  A  consists  of  l's  and  0's,  Xh  consists  of  2's  and  0's  and  A  /V— A 
(remembering  that  %  has  l's  in  every  position)  differs  from  A  only  in 
that  the  0's  in  A  are  replaced  by  -l's  in  H.A-A  .  For  example,  a  aTI^X 
incidence  matrix  A>  for  ,«r-  7^  4  -^  ?  is 

10  10  10  1* 


(3-1-17)  At  - 


0  110  0  11 
110  0  110 
0  0  0  1  1  1  1 
10  110  10 
0  11110  0 
110  10  0  1 


nT  - 


A, 
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whose  inverse  is 


It  is  a  curious  fact  that  if  we  border  (3-1-18)  with  a  column  of  -l's  and  a  row 
of  -l's  that  we  obtain  a  Hadamard  matrix  j-l-g  where  the  usual  roles  of  -1  and 
+1  have  been  interchanged  in  that  -l's  appear  exclusively  in  the  first  row  and 
column. 

The  main  point  we  want  to  emphasize  about  the  choice  in  (3-1-16)  that  simpli¬ 
fies  K_l  so  remarkably  is  that  Ryser(as";  Ryser;  page  457  3  has  shown, 
and  we  quote,  "Another  important  specialization  of  values  of  A  is 

,  Xl  =  ,  X~' X  .  Such  a  configuration  is  equivalent 

to  a  Hadamard  matrix  of  order  We  summarize  Ryser's  result  in 

Figure  5. 


FIGURE  5 

EQUIVALENCE  THEOREM  OF  RYSER 

Comparing  Figure  4  and  Figure  5  one  might  suspect  the  existence  of 
Plotkin  Maximal  Codes  distinct  from  those  obtainable  by  the  Bose,  Shrikhande 
scheme.  That  such  is  not  the  case,  that  indeed  one  could  hook  the  configura¬ 
tion  in  Figure  4  and  Figure  5  together  by  equivalence  arrows  is  shown  by 
the  following  reasoning.  Assume  the  existence  of  a  aT^  Xz.}  N  configuration 
with  an  incidence  matrix  \  where  ,  Xl.  -  ,  X~  -t  .  Then 

the  matrix  \  is  a  matrix  of  zeros  and  ones  which  we  assert  is  an  inci- 
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dence  matrix  of  a  'k-//)  X^  configuration  where  w's  4-fc-/  ,  >&•=•  Xt~l  , 

-  •te-l  .  To  see  this  we  calculate,  using  (3-1-10),  that 


(3-1-19)  (<i-A) 


Ls^-  -  s  4-  a  at 


—  H^S  -4z,£  +•  ('k-A)X  4-  X^ 

—  (4.-X)  X  +-  \/r  -  x$  £ 

=  (Jk'-x')  x  4-  xV 

so  that 

(3-1-20)  S  X'  -  >T~  ft.lt- X  -  -fc-» 

t  4'  ;  Xrfc  -  I  si»vcfc  -k/  —  X/—  4c-  X  =•  "fc" 

which  satisfies  the  equivalence,  since,  of  course,  /vT^  —  4 -t  —  |  .  In 

essence  the  configurations  of  Bose, Shrikhande  and  Ryser  are  related  merely 
by  interchanging  zeros  and  ones. 


3.  2  A  MULTIPLEXING  TECHNIQUE 

The  simplicity  of  the  equation  (3-1-15),  exhibiting  the  inverse  matrix  Ai  *, 
suggests  to  us  a  multiplexing  technique  that  has  sufficient  arbitrariness  in 
its  structure  to  consider  its  use  for  an  anti-jamming  system.  Fixing  /*“  , 
which  specifies  the  number  of  channels  to  be  multiplexed  in  a  given  applica¬ 
tion,  one  looks  for  appropriate  solutions  of  (3-1-9),  i.e.  ,  permissible  values 

of  X,-k.  .  Assume  there  are  /vv  such  pairs,  call  them  (  Xp.,  {t,  ), - , 

(  X*^,  -fe~).  THEN  WE  ASSERT  THAT  THERE  EXIST  ^  MODES  OF  OPERA¬ 
TION  OF  A  MULTIPLEXING  SYSTEM  SUCH  THAT  AN  ENEMY'S  PICKING  UP 
A  TRANSMITTED  SIGNAL  IN  ONE  MODE,  AND  THEN  JAMMING  IT,  CAN  BE 
AVOIDED  BY  SWITCHING  TO  ANOTHER  MODE,  THE  SWITCHING  PROCEDURE 
BEING  SIMPLY  IMPLEMENTED.  MOREOVER,  FOR  A  DISTINGUISHED  SET  OF 
f *  'S  THERE  EXISTS  A  SIMPLEST  MODE  WHICH  COULD  BE  USED  UNTIL 
JAMMING  COMMENCES,  AT  WHICH  TIME  SWITCHING  TO  A  MORE  COMPLI¬ 
CATED  MODE  OCCURS. 


The  simplest  mode  corresponds  to  the  simplest  form  that  A~~*  can  take, 
namely,  that  specified  by  (3-1-16).  For  the  physical  realization  we  have  in 
mind  see  Figure  6. 
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FIGURE  6 

ANTI-JAMMING  (MULTIPLEXING)  REALIZATION  FOR  ^  ^  5lX  . 


The  indicated  switches  act  synchronously  and  serve  as  gating  functions.  The 
low  pass  filters  (L.P.F.  )  eliminate  the  side  bands  associated  with  gating.  In 
addition  they  are  assumed  to  be  sufficiently  narrow  band  that  the  filtered  ver¬ 
sion  of  any  of  the  signals  to  be  transmitted,  say  X,  ,  will  not  change  during 
rf  sequential  settings  of  the  gating  functions.  This,  of  course,  relates  the 

necessary  switching  rate  to  the  allowed  band  width  of  the  signals  X|  ,  - , 

to  be  transmitted. 

The  proposed  operation  of  the  switches  is  as  follows.  Instead  of  transmitting 

Xi  , - ,  trr  sequentially  on  the  common  path, we  transmit  the  "composite" 

sequence  , - ,  where 

(3-2-1)  |  =  A  X 

and  A  is  an  incidence  matrix  of  a  ^  (t.,  \  configuration.  Both  ^  and 
are  column  vectors  of  length  .  The  input  gating  functions,  which 
correspond  to  the  rows  of  the  A  matrix,  are  either  1  or  0.  Note  that  the 
arrangement  of  grounds  in  Figure  6  allows  the  gating  functions  to  be  1, 

-1,  or  0.  Thinking  of  the  Ai  matrix  exhibited  in  (3-1-17)  for  the  case 


Page  60 


r*  ny  A  ~  y  x  ■=■  t-  ,  we  see  for  example,  that 
(3-2-2)  -  X,  +  Vj  4-^i'  3-  %y 

Hence  an  enemy,  who  requires  that  he  understand  how  we  are  transmitting  in 
order  to  efficiently  jam  our  transmission,  must  obtain  AT1  and  apply  it  to  . 
Even  if  he  does  this  we  rely  on  our  ability  to  find  other  -fe^A  corresponding 
to  a  fixed  res'  to  switch  "modes"  of  transmission. 

Regarding  the  2. A  in  (3-1-16)  as  an  inessential  gain  factor,  we  set  the  output 
gating  functions,  which  corresponds  to  the  rows  of  the  SdN  ft*"'  matrix,  to 
either  +1  or  -1.  Again  thinking  of  the  k\  matrix  exhibited  in  (3-1-17)  we 
see,  for  example,  that  from  (3-1-18)  it  follows  that  the  output  of  the  first 
channel  on  the  right  hand  side  of  Figure  6  is  given  by 

(3-2-3)  H  “X,  ~  ^  t  +  7 

For  every  1,  0  setting  of  the  input  gating  functions  there  is  synchronously 
a  1,  -1  setting  of  the  output  gating  functions  for  the  case  h-  ,  our  simplest 
mode  of  operation. 

For  Arjl  v/e  vary  the  scheme  in  Figure  6  only  in  that  we  now  use  two 
switches  in  the  output  gating  functions.  To  see  that  this  suffices  examine 
(3-1-15)  rewritten  as 

(3-2-4)  (  4-  A)  A  -  A  —  -  h 

A* 

In  (3-2-4)  we  see  that  ( )  A  *  can  be  realized  as  a  composite  of  A  ,  i.  e.  , 
output  gating  functions  varying  precisely  in  step  with  the  input  gating  functions, 
and  a  fixed  set  of  switches,  i.e.  ,  invariant  in  time,  attached  to  the  negative 
terminal  of  each  output  channel  with  an  attenuator  of  magnitude  *■/ \v 
each.  See  Figure  7  . 


m 
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FIGURE  7 

ANTI- JAMMING  (MULTIPLEXING)  SCHEME  FOR  kf  2-X  . 

(Note  Additional  Switches  Characterized  by  Vk  Are  Required) 

We  could,  of  course,  not  make  any  distinction  and  always  use  Figure  7  ,  or 
equivalently  (3-2  4)  if  this  proves  more  convenient  in  engineering  practice. 

To  illustrate  the  above  scheme  we  write  down  a  incidence  matrix 

Av  for  ir-q  ,  ,  X=-|  ,  i.  e.  , 

0  10  10  10 
10  0  110  0 
0  0  1  1  0  0  1 

1  1  1  0  0  0  0 

0  10  0  10  1 

1  0  0  0  0  1  1 

0  0  10  110 

so  that  corresponding  to  (3-2-1)  we  now  have 


(3  2-6)  -h  *4  + 

and  corresponding  to  (3-2-4)  we  now  have 

(3-2-7)  ‘X'X,  - 
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at  the  output  of  the  first  channel  on  the  right  hand  side  of  Figure  7  . 


We  now  make  some  comments  on  the  relation  between  the  process  depicted 
by  Figure  7  and  an  orthogonal  multiplexing  technique  based  directly  on  an 
associated  Hadamard  matrix  Hh^  ,  whenever  the  >0^4 parameters  are 
such  that  an  Hh*  matrix  exists  in  the  sense  of  either  Figure  4  or  Figure  5  . 
For  example  corresponding  to  the  At  in  (3-2-5)  there  clearly  corresponds  the 
Hadamard  matrix  H^-fc  -  H-*  ■>  specified  by 


(3-2-8) 


+  +  +  +  +  +  +  + 

+  -+  -  +  -  +  - 

+  +  -_+  +  _- 

+  --  +  +  --  + 


+  + 


+  +  ---- 


+  -  +  - 

+  +  -  - 

+  -  -  + 


+  -  + 

+  + 

+  +  - 


which  has  the  property 

(3-2-9)  ~  %  X  '  Ht'  -  l-U 


so  that  Hs  is  essentially,  except  for  a  "gain"  factor,  its  own  inverse.  Hence 
if  in  Figure  7  we  omit  the  fixed  set  of  switches  and  allow  the  output  gating 
functions  to  vary  precisely  in  step  with  the  input  gating  functions,  both  sets 
of  switches  taking  the  values  +  1  in  accordance  with  the  rows  of  He,  we  have 
an  orthogonal  multiplexing  scheme.  This  orthogonal  scheme  was  jointly 
studied  by  M.  E.  Hines(/5' ;  Hines  )  and  the  author  (ja-t  ;  Segers )  in  con¬ 
nection  with  time  division  multiplexing  transmission,  without  companding, 
the  mix  of  "loud"  and  "soft"  talkers  accomplished  by  H  t  providing  improved, 
i.e.  ,  steadier  signals  for  transmission.  By  the  same  token  the  effect  of  the 
mixing  technique  on  interference  is  to  spread  noise  power  concentrated  in 
time  in  the  common  path  over  a  large  range  of  time  at  the  output  terminals. 
The  essential  generalization  of  the  scheme  of  Figure  7  over  that  of  Hines 
and  Segers  (the  engineering  was  first  conceived  by  Hines  and  the  study  of  the 
matrices  involved  was  then  the  work  of  Segers)  is  the  inclusion  of  the  fixed 
set  of  switches  supplementing  the  synchronously  varying  ones  of  the  ortho¬ 
gonal  scheme.  The  essential  mathematical  structure  underlying  the  possi¬ 
bility  of  such  a  scheme  is  the  "almost  orthogonal"  character  of  A  embodied 
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in  the  fact  that  $  is  of  rank  one.  The  possibility  of  the  utility  of  the  struc¬ 
ture  of  Figure  4  for  anti- jammining  is  clearly  dependent  on  the  number  of 
at,  X  configurations,  for  a  given  at"  ,  that  we  can  concoct. 

The  search  for  configurations  has  become  almost  a  career  in  itself 

to  many  mathematicians  and  engineers.  We  content  ourselves  with  providing 
references  to  literature  that  has.  helped  us  rather  than  attempt  a  short,  and 
consequently  inadequate,  survey.  In  a  1962  book  edited  by  Todd(xa-;  Todd; 
page  518  )  Marshall  Hall,  Jr.  contributes  a  chapter  on  "Discrete  Variable 
Problems"  and  includes  a  section  on  "Systematic  Search  for  Block  Designs". 
The  work  of  Baumert,  Easterling,  Golomb,  and  Viterbi  at  Jet  Propulsion 
Laboratory  has  been  outstanding  and  a  good  summary  up  to  1961  is  provided 
by(jL;  Baumert )  while  recent  contributions  have  appeared  in  almost  every 
issue  of  the  Research  Summaries  of  the  Jet  Propulsion  Laboratory.  The 
work  of  the  group  at  Sylvania  is  extensive  but  just  in  the  process  of  being 
published,except  for  a  1961  report,  a  goodly  number  of  whose  ten  chapters 
were  contributed  by  Turynlis";  Turyn  )  .  Recent  issues  of  the  Proceedings 
of  the  American  Mathematical  Society  have  much  relevant  information  on 
new  techniques  and  results. 

There  is  a  further  relation  of  interest  between  the  incidence  matrix  A.  of  a 
aT,  "X  configuration  and  a  Hadamard  matrix  Assuming  for  conven¬ 

ience  that  h  -  then  by  (3-1-8)  f\1' -  B  ,  where  "&  has  A.  in  the  main 
diagonal  and  X  in  all  other  positions.  We  show  below  that  TJ>  ,  which  differs 
only  by  a  constant  factor  from  a  "correlation  matrix"  whose  eigenfunctions 
and  eigenvalues  were  studied  by  Max(a°;  Max;  page  114  )  ,  has  only  two 
eigenvalues.  The  larger  eigenvalue  has  associated  with  it  an  eigenmanifold 
of  dimension  one  and  the  smaller  eigenvalue  has  associated  with  it  an  eigen¬ 
manifold  contaiing  all  other  vectors  perpendicular  to  the  first  eigenmanifold. 
Consequently,  the  uniqueness  of  the  first  eigenvector  makes  it  of  great  in¬ 
terest  as  a  candidate  for  timing  purposes.  Quite  clearly  our  assumption  of 
synchronously  operated  switches  at  input  and  output,  in  the  face  of  natural 
noise  and  jamming,  is  a  critical  one.  Indeed,  since  the  ability  of  spread- 
spectrum  systems  to  reject  noise  and  jamming  when  synchronized  is  in¬ 
herently  superior  to  their  ability  when  synchronization  is  being  established, 
it  is  not  an  exaggeration  to  say  that  a  major  problem?if  not  the  major  problem, 
is  to  establish  initial  synchronization. 

Before  plunging  into  mathematical  detail  on  the  ideas  outlined  in  the  last 
section  we  first  simplify  the  implementation  in  Figure  7  so  as  to  put  the 
essential  question  in  better  perspective.  For  instance, using  the  matrix 
in  (3-2-5)  as  a  model, the  switch  on  channel  1  goes  through  the  sequence  of 
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values 


(3-2-10) 
and  the  s 

(3-2-11) 


and  the  switch  on  channel  2  goes  through  the  sequence 

I  I  I 


This  is  hard  to  implement;  instead  we  assert  the  possibility  of  using  com¬ 
binations  of  resistors,  say,  unit  resistors  connecting  %-t,  to  switch 

position  &  ;  unit  resistors  connecting  V,  ;  xf  to  switch  position 

jiv  5  etc.;  finally,  unit  resistors  connecting  Xy,  "*£  to  switch  posi¬ 

tion  $7  ;  and  the  usual  commutator  switch  to  gather  all  the  above  sequen¬ 
tially  from  $>!  , - ,  ^7  . 


Since  we  also  need  fixed  connections  to  negative  terminals  on  the  output  in 
Figure  7  and  we  might  want  to  reverse  the  process,  i.e.  ,  put  fixed  con¬ 
nections  to  negative  terminals  on  the  input,  the  most  general  structure  of 
interest  is  gotten  by  replacing  the  "inner  core"  of  Figure  7  ,  i.e.  ,  the 
structure  of  Figure  8  . 


FIGURE  8 

"INNER  CORE"  of  FIGURE  7 
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We  replace  the  inner  core  in  Figure  8  by  the  structure  Figure  9. 


FIGURE  9 

ALTERNATE  CIRCUITRY  WITH  COMMUTATORS, 
i  RESISTANCE  MATRICES  AND  VARIABLE  DELAY 

j 

■  The  transmission  properties  of  Figure  7  and  Figure  9  are  identical. 

We  have  merely  replaced  the  complicated  switching  pattern  by  interconnection 
{  matrices  plus  simple  switches  and  indicated  a  variable  delay  in  the  common 

j  path  to  emphasize  the  problem  of  putting  the  information  in  the  correct  output 

channel.  We  now  return  to  mathematical  considerations  concerning  the 
;  matrix  B  . 

i 

( 

Theorem:  The  matrix  "B  of  order  in  (3-1-8), 

is  specified  by 

(3-2-10)  A*  «  B  =  o- -X)  i  -P  x  £  j  A^AT 
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This  matrix  has  only  two  distinct  eigenvalues,  Az.~  X 
and  Ar~'3  X  •  The  eigenvector  associated  with 

the  larger  eigenvalue  is  the  column  of  consisting 
entirely  of  l's.  The  t<r-\  remaining  columns  of  H 
are  eigenvectors  associated  with  I  ~  • 


Proof;  Clearly  we  have  that 


V 

X 

' 

~  1 

- 1  “ 

_>• 

_l  - 

u 

so  half  of  the  statement  is  established.  Now  consider 
(3-2-12)  B  X  - 

i.e.,  try  to  find  eigenvectors  X  associated  with  -fe-X  .  Using  (3-2-10)  in 
(3-2-12)  we  obtain 

(3-2-13)  X)  It  +-  X 


and  since  X^o  ,  (3-2-13)  yields 
(3-2-14)  £lc  ~  O 


Since  the  rank  of 


is  one, (3 -2-14)  reduced  to  the  single  equation 


(3-2-15)  X,  i - +-  X  /r  •  aT- 

which  is  satisfied  by  the  nt-|  columns  of  H  -  H  <j-f  that  are  not  all  l’s. 
Clearly,  there  are  no  more  than  eigenvectors.  Q.  E.  D. 


The  ratio  of  eigenvalues 

(3-2-16)  £  ~  M, 

k  -  X  X 

where  we  have  used  (3-1-9)  to  simplify  the  numerator,  is  a  measure  of  how 
much  "magnification"  the  matrix  &  produces  on  the  distinguished  vector  of 
all  l's  relative  to  the  other  vectors. 


The  essential  problem  that  remains  is  to  relate  the  eigenvalues  and  eigen¬ 
functions  of  A  ,  the  square  root  of  3  ,  to  those  of  3  .  Here  we  have  an 


ambiguity  in  taking  the  square  root  of  eigenvalues.  Granting  that  this  can  be 
resolved  there  remains  the  problem  of  how  precisely  in  the  mechanism  of 
transmission  to  reserve  the  timing  signal  so  that  the  information  process  will 
not  initiate  it,  or  noise  initiate  it,  to  give  fake  timing  information.  The  solu¬ 
tion  of  these  problems  requires  further  study. 
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